Data Engineer  

We are seeking a skilled and detail-oriented Data Engineer to join our dynamic team. The Data Engineer will play a crucial role in designing, developing, and maintaining robust data pipelines, ensuring that our data is accurate, accessible, and structured for use in analytics and business intelligence. This role requires a combination of strong technical skills, problem-solving abilities, and a deep understanding of data architecture.

1. Memiliki pengalaman minimal 5 tahun

2. Menguasi programming language:

• Phyton
• Advance SQL (subquery, row_number, rank, partition by, Indexing, Explain Plan, dan Query Tuning, Stored Procedures, Triggers, View, Materialize View, Constraints, indexing, query optimization)

3. Menguasai database systems:

• MySQL
• PostgreSQL
• Oracle
• Snowflake
• Unstructured Data

4. Menguasai data modeling dan arsitektur:

• Distributed Data Storage Systems (partitioned data storage, sorting key, SerDes, data replication, caching and persistence)
• Distributed Data Processing Systems (partitioning, predicate pushdown, sort by partition, maintaining size of shuffle blocks, window function, leveraging all cores and memory available in the cluster to improve concurrency)
• Stream Data Processing (Real-time, Stream and Batch Processing)

5. Menguasai Tools:

• Data Pipelines and Automation (Airbyte)
• Data ingestion in Message Queues
• Data Wrangling operations – Pandas,numpy,re
• Data Scraping – requests/BeautifulSoup/lxml/Scrapy
• Interacting with External APIs and other Data Sources, Logging
• Parallel processing Libraries – Dask, Multiprocessing
• Data engineering tools (Apache Kafka, Apache Airflow)
• Cloud Native Technologies (Serverless Computing, Virtual Instances, Containering – Docker, Orchestration -Kubernetes)
• Data Visualization (Qlikview, Tableau)