Job Description

Description

Role Overview:

As a Data Engineer, your primary role is to support and enhance our current ETL processes, ensuring the efficient ingestion, cleansing, and transformation of data from various sources. You will also play a key role in the development and maintenance of our data lake and data pipelines, ensuring that our organization can access and analyze data effectively. Additionally, your responsibility includes meticulous documentation of data processes to ensure transparency and compliance with best practices.

Key Responsibilities:

Data Ingestion & Integration

• Integrate data from multiple sources, including databases, APIs, and external data providers.
• Design, build, and maintain robust data pipelines to ingest, process, and transform data from various sources into usable formats.
• ETL (Extract, Transform, Load): Create ETL processes to clean, transform, and load data into our data warehouse.
• Implement data quality checks and validation processes to ensure data accuracy and consistency.

Data Warehousing

• Develop and manage data warehousing solutions to store and organize large datasets efficiently.
• Design and implement data models that facilitate efficient querying and reporting.
• Monitor and optimize data pipelines and databases for performance and scalability.

Documentation

• Maintain thorough documentation of data processes, schemas, and workflows

Collaboration

• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and provide data solutions to meet their needs.

Requirements

Required Skills and Qualifications:

• Degree in Engineering or Information Technology.
• Proven experience (minimum 1-2 years) in data engineering
• Proficiency in SQL or Python
• Proficiency in Power BI
• Proven expertise in extracting data from MySQL, Microsoft SQL Server, MongoDB, Google Big Query, Snowflake, Data Lakes, APIs, Log Files and other diverse data sources
• Solid understanding of data management and database principles.
• Strong problem-solving skills and a detail-oriented approach to handling data.
• Exposed to software development lifecycle
• Experienced in DevOps practice

Job Requirements