Image Credit: medium.com
Figure 9.1 Map, Discover, Evaluate - Technology in TED-ERA fan
Technology Best Practices to Reduce Data Science Project Failure Rate Executive Briefing
But Are Efforts and Disciplines in the Technology Dimension Enough?
A data scientist working with multiple business partners on a weekly basis may receive many well-intentioned data science requests and project proposals. It becomes essential to get into the habit of clarifying the deeper questions behind questions using Domain Knowledge.
In this blog, we introduce the “D” for the Domain Knowledge portion in the TED-ERA fan, and discuss three additional key steps: Clarify, Aware, and Navigate.
Process: Clarifying the Business Context of Opportunities
In organizations large and small, technical functions are set up to execute the organizational vision and mission. Data science as a technical function must align its charter to the overall organization’s strategic directions.
Clarifying the business context goes deeper than interpreting the organizational vision and mission. It involves an articulation of the maturity of the organization, the products, and their features to recognize the prime opportunities for collaborating with specific business functions at each maturity stage. Getting fewer things done better is the key.
With prime opportunities recognized, communication of the opportunities with clarity and impact to the right people at the right time is essential to align on the specific path forward. A clear project narrative must be crafted which can include the project’s description, meaning, relevance, value, and purpose, to ensure the project aligns and meets stakeholders’ expectations.
Platform: Being Aware of Data Source Nuances
With the opportunities clarified into concise project narratives, the nuances of the data sources used become important to understand. Examples of data source nuances include data biases, inaccuracies, incompleteness that inevitability exists in many domains such as web session data, geolocation data, and financial transaction data.
Taking user web sessions as an example, interactions with a website are recorded with HTTP cookies to improve personalization on future visits. Cookies are recorded on the client-side (in a browser) to maintain state within sessions and across sessions. A web session is often defined as a sequence of interactions between a client and a server, followed by periods of inactivity. This definition implies that sessions do not have explicit endpoints.
For websites with frequently updating content, robot crawler (bot) sessions could be as much as 90% of sessions, and any issues in bot detection could introduce biases in analysis. There may also be prior implementation bugs that have polluted client cookies states. If the user does not visit, the polluted cookies may persist indefinitely, causing analysis inaccuracies when they do visit.
The best-effort nature of web traffic delivery will also miss actions, causing incompleteness in data recorded. Users can also clear cookies for privacy reasons, thus triggering a loss of historical state.
Being aware and appropriately communicate to stakeholders on the limitation caused by data biases, inaccuracies, and incompleteness allows us to set more realistic expectations on achievable results from data science projects.
People: Navigate the Organizational Structure
As a data science practitioner, how many times have you been in a conversation where a product manager needs a key data science capability next week and has to launch a feature without it?
Many data science projects take weeks to set up and months to iterate through multiple phases toward their full potential. Navigating product roadmaps to plan to prepare for data science projects early is key to anticipating products and features needs to meet expectations.
The technology maturity cycle can help us anticipate not just product roadmaps, but also the business needs and common business partners to engage as a product or feature matures.
On the technology side, data science project success is also dependent on the maturity of data science practices within an organization. The Bardess Data Science Maturity Curve captures it concisely.
Figure 9.2 The Bardess Data Science Maturity Curve (source)
The Bardess Data Science Maturity Curve exposes five stages of maturity: Learning, Emerging, Functional, Integrated, and Cultural. While many organizations have functional data science teams with dedicated headcounts and defined use cases, an integrated data science practice involves platform integration with data architecture, BI tools, and internal and customer-facing apps. In the cultural stage, an organization would look to data scientists for innovation in advancing its technical and organizational needs.
Opportunities to Dive Deeper
In showcasing the relevance of domain insight in data science projects, we had dedicated an executive briefing in September at the O'Reilly AI conference in San Jose to discuss each area of concern for applying domain expertise in detail.
For the upcoming O’Reilly Strata Data & AI conference in San Jose March 15–18, we have designed and will be delivering a 3.5-hour tutorial discussing both Applying Technology and Domain Insight to AI and ML projects to increase success, with examples, questions, caveats, recommendations and outlooks. This is separate from the Technology Oversight executive briefing and more comprehensive than the executive briefings on each topic.
We invite you to join us at the upcoming executive briefing and tutorial and encourage you to continue reading this blog series on Career in Data Science. Similar to what we have done in discussing the Technology part of the fan, we plan to dive into each of the areas for domain expertise in coming blogs; however, as we are enthusiastically preparing for the March 16th tutorial and the March 17th executive briefing, please excuse a brief pause in blogging. We will return to it shortly after the conference.
If you have views, questions, or examples you would like to share, feel free to comment below or contact us directly.
About the authors:
Dr. Jike Chong is a director of data science at LinkedIn, leading a team to develop products to advance people’s careers in the context of the hiring marketplace. He previously built and grew several high-performing data science teams at Acorns, Yirendai (NYSE:YRD), and Simply Hired. Dr Chong also advises several startups and teaches at Tsinghua University.
Yue Cathy Chang is an executive leader in business development, sales and marketing for data products and services. Cathy has held strategic roles serving customers using data science- and engineering-based solutions at bluechip software enterprise companies and startups. Well-versed in tech and business, she has frequently spoken at data and AI events.
