s a Data Engineer / Data Scraping Engineer, you will be responsible for designing, developing, and maintaining data pipelines and scraping solutions to collect and process data from various sources. Your work will directly contribute to the success of our data-driven initiatives and enable data-driven decision-making.

Key Responsibilities:

• Data Pipeline Development: Build and maintain ETL (Extract, Transform, Load) pipelines to ingest, clean, and transform data from diverse sources into our data warehouse.
• Data Quality Assurance: Implement data quality checks and validation processes to ensure the accuracy and integrity of collected data.
• Implement data models and database schemas that optimize data storage and retrieval.
• Data Scraping: Develop web scraping scripts and tools to extract data from websites, APIs, and other online sources. Ensure data quality and reliability.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field
• Proven experience as a Data Engineer, with at least 3 years of experience
• Proficiency in programming languages such as Python, Java, or Scala
• Experience using a pipeline orchestrator (e.g., Temporal, Airflow)
• Strong SQL skills for data manipulation and querying.
• Extensive experience with data modeling, database design, and performance optimization techniques
• Familiarity with data storage technologies (e.g., SQL databases, NoSQL databases).
• Knowledge of data warehousing concepts and tools
• Experience with AWS or similar cloud providers
• Excellent problem-solving and communication skills
• Experience using dbt in data pipeline orchestration and processing
• Good understanding of code version control and of deployment & monitoring strategies
• Proven experience in web scraping and data engineering is a plus
• y, Engineering or related field.

• للتقديم الان