PinnedPublished inData Engineer ThingsData Formats and Compression in Data Engineering: Best Practices for CSV, Excel, JSON, Parquet, and…Photo by Killian Cartignies on UnsplashFeb 455Feb 455
PinnedPublished inData Engineer ThingsGuide to PySpark: Transformations, Techniques, and Best PracticesPySpark, the Python API for Apache Spark, simplifies big data processing with distributed computing capabilities. It supports real-time…Jan 2451Jan 2451
PinnedPublished inData Engineer ThingsDeep Dive into Data Extraction: The First Step in Data LifecycleData extraction is the process of retrieving data from various sources to be used for further analysis. The extracted data could be…Jan 3054Jan 3054
How Mobility-as-a-Service (MaaS) Companies Can Orchestrate Data Governance and AnalyticsThe Mobility-as-a-Service (MaaS) industry thrives on data — handling millions of ride requests, tracking vehicle locations, optimizing…Feb 19Feb 19
Published inData Engineer ThingsData Governance: The Real-World Playbook for Quality, Compliance, and SecurityCompanies are drowning in data but starving for insights. Effective data governance ensures that organisations can trust, protect, and use…Feb 17Feb 17
Published inData Engineer ThingsNavigating the Data Lakehouse Revolution with Apache IcebergOver the past few months, I’ve been diving deep into the world of data engineering and have discovered that Apache Iceberg is a game…Feb 1253Feb 1253
Exploratory Data Analysis (EDA) in Data lifecycleWhat is Exploratory Data Analysis (EDA)?Feb 450Feb 450
Deep Dive into Data Cleaning: Preparing Data for AnalysisWhy is Data Cleaning Important?Feb 251Feb 251
Overlap in Data Careers: Guide to Build Your Unique PathIn today’s data-driven world, job titles like Data Scientist, Data Analyst, Machine Learning Engineer, AI Engineer, and Data Engineer seem…Jan 27Jan 27
Exploring the Diverse Ecosystem of Data Engineering: One Use Case, Countless ToolsRecently, I came across an intriguing LinkedIn post. The author presented a single data engineering use case, and what followed in the…Jan 26Jan 26
Data Careers in the Age of AI Agents: Evolving, Not EndingWhy LLM Expertise, Distributed Computing Skills, and Data/AI Governance skills will enhance your career.Jan 2350Jan 2350
Use Of Streaming Data And Building AI Systems On IOT DataIoT (Internet of Things) data has revolutionised how companies operate, enabling them to gather, analyse, and act on real-time information.Jan 2221Jan 2221