Data Engineer Manager - CreditNinja
Job Overview
We are looking for a Data Engineer Manager to join a team of machine learning and data engineering experts. The hire will be responsible for managing the data engineering team as they build tooling to support analytics, extend our machine learning platform, expand and optimize our data pipeline architecture, and work with the Development team to create cross-team solutions. The ideal candidate is an experienced data engineer and data wrangler who enjoys optimizing data systems and building them from the ground up. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. Experience in analytics and statistics is a major bonus. The right candidate will have previous management experience with an engineering team and be excited by the prospect of optimizing or even re-designing, our company’s data architecture to support our next generation of products and data initiatives. The candidate should also be comfortable with leading and designing projects initiatives that have both direct customer impact and backend impact for other CreditNinja teams.
Responsibilities for Data Engineer Manager
· Coach and mentor the data engineering team: guiding, planning, and reviewing the team’s work.
· Participate in the design and building of infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Python, and AWS.
· Set the bar for the quality of technical and data-based solutions the team ships; establish and enforce code review procedures and code quality standards.
· Create and maintain optimal data pipeline architecture.
· Focus on test driven design and results for repeatable and maintainable processes and tools.
· Interface with the analytics team to build analytics tools that provide actionable insights into key business performance metrics, as well as supporting the needs of the analytics team.
· Create data-handling tools for analytics and data scientist team members that assist them in building and optimizing our decision-making process.
Qualifications for Data Engineer Manager
· Experience managing engineers and guiding a team of engineers through project planning, execution, and quality control stages.
· Strong project management and organizational skills and the ability to work independently in a fast-paced, quickly changing environment. Ability to keep up with several projects at once and understand the impact of projects within a larger system.
· Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL), as well as working familiarity with a variety of databases.
· Experience building data pipelines, architectures, and data sets from raw, loosely structured data.
· Experience building processes supporting data transformation, data structures, metadata, dependency, and workload management.
· Experience supporting and working with cross-functional teams in a dynamic environment.
· We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:
o Experience with relational SQL databases, including Postgres and MySQL.
o Experience with object-oriented design in Python
o Experience with data pipeline and workflow management tools
o Experience with AWS cloud services: EC2, RDS, Redshift, Glue, S3
· Nice to haves:
o Strong analytic skills and understanding statistical methodologies
o Experience building machine learning models
o Experience handling data from acquisition to usage in models
o Experience building and maintaining RestAPI systems, Flask apps, and state machines
o Experience with continuous integration, especially in a data science context
o Experience with Ruby (on Rails)