Skill Set :

1. Sound knowledge on Azure cloud platform.
2.  Strong SQL skills for data manipulation and query optimization.
3. Good knowledge of Visual Studio Code development.
4. Knowledge of data warehouse architectures, design patterns, and performance tuning techniques.
5. Familiar with Power BI or other BI reporting tools.
6. Expertise in data integration and data quality.
7. Experience in data integration and data quality.
8. Hands on experience on unified data analytics platform with Azure Databricks, Databricks Workspace user interface, managing Notebooks which use Delta Lake.
9. Excellent analytical and organization skills.
10. Excellent verbal and written communication skills.

Job Description :

1. Design and develop data warehouses and data marts to support data analysis and reporting.
2. Overall experience on Apache Spark, SQL, DBT, Databricks, Hive, HDFS, Airflow, Kafka required.
3. Develop SQL based data processing pipelines which includes, but not limited, SQL functions, tables, and views.
4. Implement defined technology best practices and guidelines.
5. Create data validation rules on source data to confirm the data has correct and/or expected values.
6. Perform data profiling of source data to identify data quality issues and anomalies, business knowledge embedded in data, gathering of natural keys, and metadata information.
7. Leverage Databricks to optimize and accelerate processing and analytics tasks.
8. Collaborate with Databricks Administrators and Platform engineers to ensure optimal performance of data processing pipelines executed on Databricks.
9. Implement data quality rules and data governance policies to ensure data accuracy and consistency.
10. Develop Azure Databricks data engineering tasks