Energizing Economics with a Financial Flair: Your Business Expert!

Prerna Sharma - Data Engineer About Me: I am a dedicated and skilled Data Engineer with extensive experience in designing, implementing, and managing scalable data solutions. My expertise lies in utilizing a variety of data engineering tools and technologies to streamline data workflows, ensure data integrity, and enable advanced data analytics. I have a strong background in working with both on-premises and cloud-based data solutions, particularly on the Azure platform.

Technical Expertise: Azure Ecosystem: Azure Synapse Analytics: Migrating data from on-premises to Azure Synapse, designing and implementing scalable data warehouses. Azure Timestream DB: Establishing Kafka connections, streaming data from Delta tables to Timestream DB. Azure Data Factory: Creating and managing ETL pipelines for data ingestion and transformation. Azure Databricks: Developing Spark applications using Python for data processing and analysis. Data Integration and Streaming: Kafka: Connecting Kafka with various data sources, streaming data to time-series databases and MLOps models. Delta Lake: Creating and updating Delta tables, implementing merge logic for data updates. Data Modeling and ETL: Designing end-to-end ETL pipelines for data integration and analysis. Implementing hashing logic to identify data differences and ensuring data consistency. Programming and Scripting: Python: Developing data processing scripts, working with PySpark for big data processing. SQL: Writing complex queries, creating hierarchical data structures, and performing data transformations. Data Governance and Security: Ensuring data security and compliance with industry standards and best practices. Implementing robust data governance frameworks. Projects and Achievements: Successfully migrated large datasets from on-premises to Azure Synapse, enhancing data accessibility and performance. Developed and maintained data pipelines that integrate streaming data from Kafka to Azure Timestream DB, supporting real-time analytics. Created a hierarchy of managers and employees using PySpark, enabling better organizational data management. Applied innovative hashing logic to improve data accuracy and consistency in data processing workflows. Designed scalable CI/CD pipelines for data integration, reducing deployment times and improving efficiency. Personal Attributes: Strong problem-solving skills with a keen eye for detail. Ability to work collaboratively in cross-functional teams. Excellent communication skills, enabling effective interaction with stakeholders.