The leaves are changing, Pumpkin Spice Lattes are back, and your neighbor has busted out his 12-foot lawn skeleton. This can only mean one thing: it's spooky season. 🎃
For data leaders, nothing is scarier than an email from your CEO asking "what happened to this dashboard?" or a breaking schema change.
The good news? When it comes to scaling data reliability, we've got you covered. So sit back, relax, and read on for the latest and greatest in data quality and observability, including best practices, real-world use cases, binge-worthy podcasts, and more.
Scroll down for:
- 📚 Data quality tips for busy data leaders
- 🔥 This month’s hot, medium, and mild takes on data + AI
- 👯 Info on our upcoming events, including the Snowflake World Tour, Databricks World Tour, AWS re:invent, and IMPACT World Tour
- 💡Our meme of the month
What’s new?
What we’re writing about:
- The “Who Does What” Guide to Enterprise Data Quality. Operationalizing data quality can be daunting. We don’t claim to have all the answers, but we share some suggestions for how to operationalize incident management at enterprise scale.
- The 5 Data Rules You Should Never Write Again. If you’re a data analyst tired of writing manual rules, this is an article you won't want to skip. From validity rules and dimension drift, here are some checks you can — and should — easily automate.
- How WHOOP Built and Launched a Reliable GenAI Chatbot. For most data teams, driving real-world value with GenAI is easier said than done. In our latest, we highlight how Matt Luizzi and the team at WHOOP created an LLM-powered chatbot to drive data accessibility for his stakeholders.
What we’re talking about:
- Is Data Quality the Biggest Threat to Humanity? Monte Carlo CEO Barr Moses and Olga Maydanchik, Data Quality Expert, joined host Cindi Howson on The Data Chief podcast to dig into what expert Larry English once described as the biggest threat to humanity: data quality.
- How SurveyMonkey Sharpens dbt Performance and Governance with Data Observability. Samiksha Gour, Senior Data Engineering Manager at SurveyMonkey, discusses how she leverages data observability to reduce Snowflake spend and improve dbt model performance. Tune into the replay of her dbt Coalesce 2024 session.
- Bloomberg’s Tech Disrupters: Monte Carlo CEO on Honing Knotty Corporate Data. Barr Moses joined Sunil Rajgopal, senior software analyst at Bloomberg Intelligence, to discuss the intricacies of data-dependent intelligence. Say that five times fast.
What’s hot? 🌶️
We share one hot take, one medium take, and one mild take on what’s happening in the data space. Can you handle the heat?
- Hot Take: In the era of AI, every data steward is responsible for data quality. At least that’s what this recent article in CIO Magazine, AI’s Data Tsunami: Why Your Data Stewardship Needs an Overhaul, suggests. We couldn't agree more.
- Medium Take: If you’re tired of talking about AI, join the growing conversation around data product marketplaces. ThoughtSpot’s recent article, Governing Data Products Using Fitness Functions, is an interesting read on scaling data governance techniques in a data mesh.
- Mild Take: Synthetic Data is Like Diet Soda.That’s what the MIT Laboratory for Information and Decision Systems (LIDS) recently said when they launched their Synthetic Data Vault. “To be effective, it has to resemble the ‘real thing’ in certain ways.” What do you think?
Let’s meet up!
Every month is a busy month when it comes to events in the data space – and that’s just the way we like it. Catch us at these upcoming events:
- IMPACT 2024: The Data Observability Summit: Join us on November 14 for IMPACT Virtual Summit to hear from Allan Lichtman, creator of the Keys to the White House prediction model, and Allyson Felix, Olympic gold medalist in track and field, and more on the increasing importance of trusted data and AI to power growth and innovation.
- IMPACT World Tour: We’re taking IMPACT on tour with in-person events in a city near you! First, we’re headed to Atlanta, London, San Francisco, Los Angeles, Denver, and Minneapolis. See you there?
- Snowflake World Tour: We’ll be at the Snowflake World Tour in Stockholm on October 17! Join us for a fireside chat with Pedro Sá Martins, Head of Data Engineering at Outsystems, who will discuss the evolution of his team's data observability journey.
- Databricks Data + AI World Tour: Join us at our upcoming stops in Los Angeles, Chicago, and Munich to learn how Databricks and Monte Carlo are pioneering data reliability for the lakehouse!
- AWS re:Invent: Viva las Vegas! Or should I say: viva AWS re:invent! Mark your calendars now for AWS Re:Invent, happening December 2-6 in Vegas, and be sure to stop by booth Monte Carlo booth #667.
What we’re reading:
What’s better than working with data? Reading about data! Here are a few articles from across the industry that piqued our interest.
- DataMesh: How Uber laid the foundations for the data lake cloud migration, from Uber’s Engineering Blog. A great deep-dive read on how Uber architected the layers of abstraction in their data mesh.If your company is embarking on (or currently navigating) their data mesh journey, be sure to give it a read.
- Evaluating Data Observability Tools: A Comprehensive Guide, from Data Engineering Weekly. A detailed guide (and RFP template) that can serve as a useful tool if you’re considering starting the evaluation process for a data quality or observability platform. If reliability is on the roadmap, this resource is a must-have.