Junior Data Engineer

1. Minggu 1: Onboarding tim AI & Data Modelling: setup development environment (Python, SQL client, Git, orchestrator access), review data-protection guidelines (GDPR / UU PDP) untuk employee data, memahami arsitektur warehouse yang ada secara high-level. 2. Minggu 2: Inventarisasi seluruh sumber HR data: HRIS modules (employee master, position, attendance), vendor APIs (payroll provider, ATS, LMS, benefits), flatfile drops, attendance devices. Untuk setiap sumber: owner, extract method, cadence, volume, sensitivity class. 3. Minggu 3: Interview source owners (6–8 orang) untuk memahami extract cadence actual, schema change history, known data quality issues, dan tribal knowledge yang perlu didokumentasikan. 4. Minggu 4: Draft layered warehouse design (raw / staging / curated), definisi conformed dimensions (dim_employee, dim_position, dim_org_unit, dim_date), draft naming convention. Presentasi Warehouse Architecture Blueprint ke AI Engineer untuk sign-off.

1. Minggu 1: Setup repository structure dengan code review workflow, orchestrator environment (dev + prod), secret management untuk credentials. Scaffold pipeline template agar reusable antar sumber. 2. Minggu 2: Build ingestion pipeline sumber pertama (HRIS): API authentication, extract logic (handle pagination & rate limit), raw layer load dengan metadata (ingested_at, source_batch_id). Design & test watermark untuk incremental loading. 3. Minggu 3: Build ingestion pipeline sumber kedua (payroll): typically monthly cadence, handle schema differences dari HRIS, idempotency testing (re-run di non-prod tidak boleh duplicate atau corrupt data). 4. Minggu 4: Scheduling di production, setup logging & basic alerting untuk failed runs, dokumentasikan runbook per pipeline (cara re-run, cara backfill, known failure modes).

1. Minggu 1: Setup dbt project (atau framework setara), project structure, define macros untuk reusable logic. Build staging layer untuk HRIS + payroll (rename, type cast, null handling, dedupe). 2. Minggu 2: Build conformed dimensions: dim_employee dengan SCD Type 2 untuk effective-dated records (promotion, department transfer tetap bisa di-track historically), dim_position, dim_org_unit, dim_date. 3. Minggu 3: Tambah ingestion pipeline ATS + staging models + fact tables (fact_hire, fact_attrition). Setiap fact table punya documented grain (satu row = satu hire event, satu row = satu termination event). 4. Minggu 4: Tambah ingestion LMS, Mid-Term Review: demonstrasi warehouse models + lineage ke AI Engineer dan Data Scientist. Dokumentasi semua grain & key relationships.

1. Minggu 1: Define test taxonomy: schema tests (not null, unique, accepted values), freshness tests (max_loaded_at vs expected cadence), referential integrity tests (foreign keys resolve), row count anomaly detection. 2. Minggu 2: Business rule tests: "no active employee without valid cost_center", "payroll amount must match contract band", "hire_date <= termination_date", "position effective_date tidak overlap dalam SCD". Mapping tiap test ke owner. 3. Minggu 3: Setup alerting routing (Slack / email) ke named owners — bukan ke channel umum yang biasanya di-mute. Define severity levels (warning vs critical) dan expected response time per severity. 4. Minggu 4: Query performance review: identifikasi 3 query / job tersulit, tune dengan indexing / partitioning / incremental materialization. Dokumentasikan before-after metrics (runtime, bytes scanned).

1. Minggu 1: Classification sensitive columns per sensitivity class: PII (NIK, nama, alamat), compensation (salary, bonus), performance (rating, warning). Define masking strategy per class (full mask, partial mask, tokenization). 2. Minggu 2: Implementasi masking / pseudonymization di layer analytics untuk compensation & identity data. Setup column-level access controls (role-based: analyst, HR BP, HR leadership). 3. Minggu 3: Define & implementasi retention rules sesuai policy (contoh: raw payroll retention 7 tahun, exit interview retention 2 tahun). Populate data dictionary (owner, description, source, grain, sensitivity class, retention) untuk semua production tables. 4. Minggu 4: End-to-end lineage documentation: visual lineage dari source → raw → staging → curated → consumer. Audit sensitivity classification di semua layer (pastikan sensitive column tidak bocor ke downstream yang access-nya lebih luas).

1. Minggu 1: Discovery session dengan HR team: inventarisasi recurring manual extracts / spreadsheet processes (contoh: weekly headcount report, monthly attrition tracker, quarterly compensation audit). Prioritisasi berdasarkan time saved + error reduction. 2. Minggu 2: Otomatisasi 3 manual processes prioritas: pipeline + scheduled delivery (email / shared drive / BI dashboard). Validasi output dengan HR owner yang sebelumnya mengerjakan manual. 3. Minggu 3: Menyusun Pipeline Maintenance SOP: cara menambah source baru, handling schema change dari vendor, backfill procedure, incident response. Knowledge transfer session dengan tim internal. 4. Minggu 4: Final Defense: presentasi hasil + Automation Impact Report (time saved per week, error reduction) di hadapan AI Engineer dan HR Leadership. Penutupan administratif pada portal MagangHub Kemnaker.