Lead Data Engineer

Country:

Location:

Role: Lead Data Engineer

About the Role: Experienced Data Science & Analytics professional. Creating data mining architectures/models/protocols, statistical reporting, and data analysis methodologies to identify trends in large data sets. Analysis may be applied to various areas of the business (e.g., Market Economics, Supply Chain, Marketing/Advertising, Scientific Research, etc.)

Primary responsibilities: 

• Architect and implement scalable and robust data infrastructure on AWS to support AI and Advanced Analytics use cases

• Develop and optimize data pipelines for efficient data processing and integration using AWS services and DataIku

• Assist in formulating data governance standards and ensure data quality, integrity, and compliance with security policies

• Collaborate with data scientists and MLOps engineers to design, build, and maintain a feature store for rapid AI model development

• Implement monitoring systems to track data infrastructure performance and make improvements as needed

• Manage and optimize AI CoE data accounts, warehouses, and tables on AWS

• Build and maintain data connectors to AI systems

• Create templates and best practices for establishing new data connections and formats

• Formulate and implement best practices for data storage and access for AI tools

• Develop standards around optimization of data transformations

• Stay updated with the latest advancements in data engineering, particularly within the AWS ecosystem, and drive the adoption of innovative techniques

Experience and Skills Required:

• 7+ years of experience in data engineering, with a focus on building and maintaining data infrastructure

• Extensive experience with AWS data services such as S3, Redshift, Glue, EMR, and Athena

• Proficiency in Dataiku for data integration, preparation, and pipeline management

• Strong skills in programming languages such as Python and SQL

• Experience with big data technologies and ETL processes

• Familiarity with data governance principles and data security best practices in cloud environments

• Strong understanding of AI and machine learning concepts and their data requirements

• Excellent problem-solving and analytical skills

• Ability to work independently and manage multiple priorities in a fast-paced environment

• Strong communication skills with the ability to explain complex technical concepts to both technical and non-technical stakeholders

• Experience with CI/CD practices and DevOps principles

• Familiarity with data modeling and database design

• Knowledge of data visualization tools and techniques is a plus

• Ability to mentor and guide junior data engineers as the team grows

Qualifications:

• B.E/B.Tech/MCA/M.Tech qualification

• AWS and Dataiku Certified preferred, but not mandatory

Additional Comments:

This individual will be self-directed, highly motivated, and organized with strong analytical thinking and problem-solving skills, and have an ability to work on multiple projects and function in a team environment.

Benefits:

• We are committed to offering competitive benefits programs for all of our employees, and enhancing our programs when necessary.

• Have peace of mind and body with our health insurance

• Make yourself a priority with flexible schedules and leave Policy

• Drive forward your career through professional development opportunities Achieve your personal goals with our Employee Assistance Program.

Our commitment to you:

Our greatest assets are the expertise, creativity and passion of our employees. We strive to provide a great place to work that attracts, develops and retains the best talent, promotes employee engagement, fosters teamwork and ultimately drives innovation for the benefit of our customers. We strive to create an environment where you feel that you belong, with diversity and inclusion as the engine to growth and innovation. We develop and deploy best-

in-class programs and practices, providing enriching career opportunities, listening to employee feedback and always challenging ourselves to do better. This is The Carrier Way.

Join us and make a difference.

Apply Now..

Carrier is An Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status, age or any other federally protected class.

Job Applicant's Privacy Notice:

Click on this link to read the Job Applicant's Privacy Notice