16 Data Scientist Interview Questions: A/B Tests, Stats, SQL, ML

Last updated: October 2026

Quick Answer

Data scientist interview questions test whether you can turn an ambiguous product question into a sound decision: reason about probability and uncertainty, design and read an experiment without fooling yourself, pull the data yourself in SQL, build and honestly evaluate a model, pick the metric that matters, and explain all of it to people who do not care about p-values. Rehearse the experiment and modeling answers aloud in OphyAI Interview Practice; Interview Copilot gives real-time answer suggestions in live video rounds.

#	Question	What the interviewer is testing
1	Explain a p-value to a product manager	Statistical intuition without jargon
2	8 heads in 10 flips: is the coin biased?	Hypothesis testing from first principles
3	A test for a rare condition comes back positive. How likely is the condition?	Bayes’ rule and base rates
4	Design an A/B test for a new feature	Unit, metric, power, duration, guardrails
5	How many users do you need?	Sample size and minimum detectable effect
6	The test is significant on day 3. Can you stop?	Peeking and sequential testing
7	The result looks too good. What do you check?	Sample ratio mismatch, novelty, bugs
8	Users in treatment and control interact. Now what?	Interference and cluster designs
9	Write a query for day-7 retention by signup cohort	Cohort SQL and metric definitions
10	Engagement up 3%, ad revenue down 1%: launch?	Metric trade-offs and judgment

What data scientists do and earn in the US

According to the US Bureau of Labor Statistics (BLS), data scientists earned a median of $120,230 a year ($57.80 an hour) in May 2025, and about 275,600 people held the job in 2025. BLS projects employment to grow 35% from 2025 to 2035, much faster than average, with about 24,800 openings a year. The typical entry-level education is a bachelor’s degree. BLS lists the core duties as collecting and analyzing data, building and validating models, and making recommendations to stakeholders, which is what the interview tests.

What first-party company pages say about the process (checked October 2026):

Employer	What its own careers page says
Capital One	Data science candidates complete a take-home Data Science Challenge before Power Day, focused on “data and engineering analytics, written communication and predictive modeling.”
Pinterest	The machine learning and data science new grad path runs CodeSignal, a 30-minute phone screen, a hiring manager review, 3-4 technical interviews, then a 30-minute call with interested managers (page dated April 2024).
Meta	A recruiter conversation, initial screening and a full loop; the page does not break out data science rounds. Candidate-reported loops for product analytics list analytical execution, analytical reasoning, technical skills and behavioral rounds.

Titles vary: product analytics roles lean on SQL, experiments and metrics; machine learning roles lean on modeling and coding. Ask the recruiter which track you are on.

The Questions, With Sample Answers

The sample answers are templates for a fictional candidate with two years of product analytics and modeling experience; numbers inside them are illustrative, so swap in your own projects. For “tell me about a time” questions, use the STAR method. For every technical question, say your assumptions out loud before you answer: interviewers grade the reasoning more than the final number.

1. “How would you explain a p-value to a product manager?”

Why they ask it: P-values are the most misused number in industry. The interviewer wants to know you understand what one does and does not mean, and that you can say it in plain English.

Sample answer: “I’d say: assume the new checkout button has no real effect. If that were true, how surprising would the difference we saw be? The p-value is that surprise level. A p-value of 0.03 means that if the button truly did nothing, we would see a gap this large or larger about 3% of the time just from random variation. It is not the probability that the button works, and it says nothing about whether the effect is big enough to matter. So I always pair it with the effect size and a confidence interval, for example ‘conversion rose 0.4 points, plausibly between 0.1 and 0.7,’ and then we decide whether that range is worth shipping.”

What weak answers look like: “It’s the probability the null hypothesis is true,” or “below 0.05 means it worked.” The American Statistical Association’s statement on p-values lists both as misreadings.

Prepare with OphyAI: explain this to a non-technical persona in Interview Practice and check the transcript for jargon you did not define.

2. “You flip a coin 10 times and get 8 heads. Is it biased?”

Why they ask it: A small, exact problem shows whether you can set up a hypothesis test without a library and whether you think about one-sided versus two-sided questions.

Sample answer: “My null is a fair coin, p equals 0.5. The chance of 8 or more heads in 10 fair flips is the sum of 10 choose 8, 9 and 10, which is 45 plus 10 plus 1, or 56, over 2 to the 10th, 1,024. That is about 5.5%. Since ‘biased’ could mean either direction, I’d double it for a two-sided test, so about 11%. At a 5% threshold I would not call the coin biased. Ten flips also gives very little power, so if the decision mattered I’d collect more data first.”

What weak answers look like: Saying “80% is far from 50%, so yes,” or computing only P(exactly 8). Both skip the actual test.

Prepare with OphyAI: Interview Practice asks follow-ups, so let it push you on “what if it were 80 heads in 100?“

3. “A test for a rare condition is 90% sensitive with a 5% false positive rate. 1% of people have the condition. Someone tests positive. How likely is it they have it?”

Why they ask it: Base-rate neglect breaks fraud models, spam filters and alerting systems. This checks Bayes’ rule and your instinct to anchor on prevalence.

Sample answer: “I like to use 10,000 people so the numbers stay concrete. 1%, or 100 people, have the condition, and 90 of them test positive. Of the 9,900 who don’t, 5%, about 495, also test positive. So 585 people test positive and 90 of them actually have it: 90 divided by 585 is about 15%. Even with a decent test, most positives are false because the condition is rare. In a product setting this is why a fraud model with a low false positive rate can still flood the review queue, and why I’d look at precision at the threshold we actually use, not just the overall accuracy.”

What weak answers look like: Answering “90%,” or reciting the formula without plugging in numbers.

Prepare with OphyAI: practice the 10,000-people framing in Interview Practice until you can do it without paper.

4. “How would you design an A/B test for a new feature?”

Why they ask it: This is the core of most product data science loops. They want a full design: hypothesis, randomization unit, metrics, sample size, duration and decision rule, set before launch.

Sample answer: “Say the feature is saved searches in a shopping app. First the hypothesis: saved searches increase repeat purchases. I’d randomize by user, not session, so one person sees one experience. Primary metric: purchases per user over the test window. Guardrails: page load time, unsubscribe rate and revenue per user, so a win on one metric cannot hide damage elsewhere. Then I’d agree a minimum detectable effect with the PM, compute sample size, and run at least one full week, ideally two, to cover weekday patterns. I’d write the decision rule down in advance, for example ‘ship if the primary metric is significant and no guardrail drops more than 0.5%,’ and check assignment balance on day one.”

What weak answers look like: Jumping straight to “run it and check significance,” with no unit, no guardrails and no pre-agreed decision rule.

Prepare with OphyAI: run a case-format mock in Interview Practice and practice saying the design in that order: hypothesis, unit, metrics, power, duration, decision.

5. “How do you decide how many users you need for an experiment?”

Why they ask it: Underpowered tests waste weeks and produce noise. The interviewer wants to see you connect baseline rate, effect size, variance and traffic.

Sample answer: “Sample size depends on four things: the baseline rate, the smallest effect worth detecting, the significance level and the power. A handy approximation for 5% significance and 80% power is n equals 16 times the variance divided by the effect squared, per group. If baseline conversion is 10%, the variance is 0.1 times 0.9, or 0.09. To detect a one-point absolute lift, that is 16 times 0.09 divided by 0.0001, about 14,400 users per group. If 4,000 eligible users arrive each day, split evenly, that is roughly a week, and I’d round up to two full weeks. If that is too slow, we either accept a larger detectable effect or use variance reduction such as adjusting for pre-period behavior.”

What weak answers look like: “Until it’s significant,” or a number with no link to baseline rate or effect size.

Prepare with OphyAI: rehearse the arithmetic aloud in Interview Practice; saying numbers clearly under pressure is a skill of its own.

6. “Your test hit p < 0.05 on day 3 of a planned 14-day run. Can you stop and ship?”

Why they ask it: Peeking is the most common experiment mistake in industry. They are checking whether you know why repeated looks inflate false positives and what to do instead.

Sample answer: “Not on the fixed-horizon test we planned. A standard test controls the false positive rate only if you analyze once, at the end. If we check every day and stop the first time we cross 0.05, each look is another chance for noise to cross the line, so the real false positive rate ends up well above 5%. Day 3 also misses weekly patterns and novelty effects. If the team wants the option to stop early, the right move is to plan for it up front with a sequential design. Spotify’s engineering team, for example, has written that its experimentation platform uses group sequential tests, which allow planned interim looks while keeping the error rate controlled. Otherwise, we let it run the full 14 days.”

What weak answers look like: “Yes, it’s significant,” or “No, because the sample is small” without naming the multiple-looks problem.

Prepare with OphyAI: let Interview Practice play a PM who wants to ship today, and practice holding the line politely.

7. “The test shows an 8% lift in conversion. What do you check before you believe it?”

Why they ask it: Big wins are usually bugs. Senior data scientists distrust surprising results first and celebrate second.

Sample answer: “First, sample ratio mismatch: if we planned 50/50 and got 52/48 with large samples, the assignment or logging is broken and the result is not trustworthy. Second, I’d confirm the metric is computed the same way in both groups, for example that treatment isn’t logging duplicate purchase events. Third, I’d segment by platform and new versus returning users; if the lift is all in one app version, that may be a bug, not a feature. Fourth, novelty: I’d plot the daily effect and see whether it decays. Fifth, multiple comparisons: with 20 metrics, one may be ‘significant’ by chance. Only then would I estimate long-run impact, ideally with a holdback.”

What weak answers look like: Celebrating the lift, or listing checks without saying what each one would reveal.

Prepare with OphyAI: in Interview Practice, ask for a skeptical follow-up after every result you present.

8. “In a marketplace, treatment users take inventory from control users. How do you test a change?”

Why they ask it: In ride-hailing, delivery and booking marketplaces one user’s treatment affects another’s. This separates textbook knowledge from production experience.

Sample answer: “If treatment riders get a better price, they may book drivers that control riders would have used, so control looks worse and the measured lift is inflated. The fix is to randomize at a level where spillover mostly stays inside the unit. Options are cluster randomization by city or region, which reduces interference but cuts the number of units and therefore power; or a switchback design, where a whole market alternates between treatment and control over time windows, which works when effects wear off quickly. I’d pick based on how much the units interact and how fast the effect decays, and I’d be explicit with stakeholders that these designs need longer runs to reach the same precision.”

What weak answers look like: Proposing a standard user-level A/B test, or naming “network effects” without a design that addresses them.

Prepare with OphyAI: rehearse this with a delivery or rideshare example in Interview Practice before a DoorDash or Uber loop.

9. “Write a query for day-7 retention by signup cohort.”

Why they ask it: Data scientists are expected to pull their own data. Retention queries test joins, date logic and, above all, whether you define the metric before typing.

Sample answer: “First I’d confirm the definition: active on exactly day 7 after signup, or any day within the first 7? I’ll assume exactly day 7, with ‘active’ meaning any event.”

WITH signups AS (
  SELECT user_id, DATE(created_at) AS signup_date
  FROM users
  WHERE created_at >= DATE '2026-09-01'
    AND created_at <  DATE '2026-10-01'
),
returned AS (
  SELECT DISTINCT s.user_id
  FROM signups s
  JOIN events e
    ON e.user_id = s.user_id
   AND DATE(e.event_time) = s.signup_date + 7
)
SELECT
  s.signup_date,
  COUNT(*) AS new_users,
  COUNT(r.user_id) AS retained_d7,
  ROUND(100.0 * COUNT(r.user_id) / COUNT(*), 1) AS d7_retention_pct
FROM signups s
LEFT JOIN returned r ON r.user_id = s.user_id
GROUP BY s.signup_date
ORDER BY s.signup_date;

“The LEFT JOIN keeps users who never came back, and DISTINCT stops users with many events from being double-counted. I’d also exclude cohorts less than 7 days old.”

What weak answers look like: An INNER JOIN that silently drops non-returners, or no clarifying question about the definition.

Prepare with OphyAI: talk through the query aloud in Interview Practice; for more SQL drills see our SQL interview questions.

10. “A test raised engagement 3% but cut ad revenue 1%. Do you launch?”

Why they ask it: Real decisions involve trade-offs between metrics. They want structured judgment, not a formula.

Sample answer: “I’d start by asking what the company’s goal for this surface is and whether either change is noise: are both intervals clear of zero? If they are, I’d look at mechanism. If engagement rose because users see fewer ads, the revenue loss is the direct cost of a better experience, and the question is whether higher engagement brings it back through retention. I’d check longer-term signals in the test, like day-28 retention, and estimate the break-even: how much retention lift would offset a 1% revenue drop. If the numbers are close, I’d recommend launching to a holdback group to measure the long-run effect, and I’d bring leadership the trade-off in dollars, not just percentages.”

What weak answers look like: “Engagement matters more, ship it,” or “revenue is down, kill it,” without investigating why.

Prepare with OphyAI: practice giving a clear recommendation in under two minutes in Interview Practice; product interviewers grade the conclusion as much as the analysis.

11. “How would you choose a north star metric for a music streaming app?”

Why they ask it: Product sense rounds test whether you can pick a metric that tracks real user value and resists gaming.

Sample answer: “I’d start with what value means for listeners: finding music they enjoy and coming back to it. Daily active users is too shallow, since an open counts even if nothing plays. Total minutes streamed is closer but can be gamed by autoplay. I’d propose weekly active listeners with at least one ‘engaged’ session, say 30 minutes or a saved track, because it captures habit and satisfaction. I’d support it with input metrics teams can move: new playlist follows, search success rate, skips per session. Guardrails would be paid conversion and churn. Finally I’d check in historical data that it predicts long-term retention.”

What weak answers look like: Naming a single metric with no reason, or a list of ten metrics with no hierarchy.

Prepare with OphyAI: Interview Practice’s case format lets you rehearse a metric-design answer and get feedback on structure.

12. “Walk me through building a model to predict customer churn.”

Why they ask it: This covers the full modeling lifecycle: framing, labels, features, validation, metrics and deployment. The interviewer listens for business framing first.

Sample answer: “First I’d agree what churn means, for example no subscription renewal within 30 days of the due date, and what action the prediction will drive, such as a retention offer. That decides the prediction window and which errors matter. Features would come only from data available before the prediction date: usage trends, support tickets, plan type, payment failures. I’d split by time, training on older months and testing on the most recent, to mimic production. I’d start with logistic regression as a baseline, then try gradient boosting. Since churn might be 5% of users, I’d evaluate with precision and recall at the offer budget, not accuracy. Then I’d run the offer as an experiment, because a good churn model doesn’t prove the offer works.”

What weak answers look like: Starting with “I’d use XGBoost,” random splits on time-based data, or no mention of what action the model drives.

Prepare with OphyAI: rehearse the lifecycle in Interview Practice’s technical format, and ask for follow-ups on validation choices.

13. “Your fraud model has 98% accuracy. Is it good?”

Why they ask it: Imbalanced classes are everywhere in industry. This tests whether you choose metrics that match the cost of errors.

Sample answer: “I can’t tell yet. If 2% of transactions are fraud, a model that predicts ‘not fraud’ every time is 98% accurate and catches nothing. I’d look at the confusion matrix, precision and recall, and the precision-recall curve, since with rare positives it is more informative than ROC. Then I’d tie the threshold to costs: what a missed fraud case costs versus what a blocked good customer costs, and how many alerts the review team can handle each day. I’d report something like ‘at 500 alerts a day, we catch X% of fraud dollars with Y% precision.’ That tells the business what they get, which accuracy never does.”

What weak answers look like: “Yes, 98% is high,” or “use AUC” with no link to business costs.

Prepare with OphyAI: practice translating one metric into a business sentence in Interview Practice; it is the habit interviewers listen for.

14. “What is overfitting, and how do you detect and prevent it?”

Why they ask it: A foundational check on model evaluation and the bias-variance trade-off.

Sample answer: “Overfitting is when a model learns noise in the training data, so it scores well in training and worse on new data. That’s the variance side of the bias-variance trade-off: more flexible models fit training data closer but generalize less reliably. I detect it by comparing training and validation performance, ideally with cross-validation, and by watching learning curves: if validation error stops improving while training error keeps falling, I’m overfitting. To prevent it, I’d simplify the model, add regularization such as L1 or L2 penalties, limit tree depth or use early stopping for boosting, and get more data if possible. And I keep a final test set I touch once, so my estimate is honest.”

What weak answers look like: A definition with no detection method, or tuning repeatedly on the test set.

Prepare with OphyAI: Interview Practice can ask conceptual ML follow-ups; practice answering in three parts: what, detect, prevent.

15. “What is data leakage? Give an example you’ve seen or could imagine.”

Why they ask it: Leakage produces models that look brilliant offline and fail in production. Experienced interviewers ask it to find candidates who have shipped models.

Sample answer: “Leakage is using information during training that wouldn’t be available at prediction time, which makes offline performance look better than it will be in production. A classic example: predicting loan default and including a ‘collections status’ field, which is only filled in after someone defaults. Subtler versions include scaling or imputing on the full dataset before splitting, which lets test-set statistics into training, and random splits on time-series data, so the model sees the future. My habits: build features with an explicit ‘as of’ timestamp, fit preprocessing on training folds only, split by time for temporal problems, and treat a suspiciously high score as a bug until proven otherwise.”

What weak answers look like: Confusing leakage with overfitting, or giving no concrete example.

Prepare with OphyAI: tell one leakage story out loud in Interview Practice; a specific example is far more convincing than a definition.

16. “Tell me about a time a stakeholder wanted a different answer than your data showed.”

Why they ask it: Data scientists are only valuable if their findings survive contact with people who had a plan. This behavioral question tests integrity and influence.

Sample answer: “(Situation) Our marketing lead was sure a new onboarding email increased activation and wanted to roll it out globally. (Task) I was asked to confirm the results from a two-week test. (Action) The raw comparison showed a 6% lift, but assignment was 55/45, a sample ratio mismatch. I traced it to a bug that excluded users with unverified emails from control. After correcting it, the lift was under 1% and not significant. I walked the marketing lead through it privately first, showed the bug, and proposed a clean rerun rather than a flat ‘no.’ (Result) The rerun showed no lift, we saved the localization work for eight markets, and the team added an assignment check to every test launch.”

What weak answers look like: Giving in to the stakeholder, or “winning” by embarrassing them in a meeting.

Prepare with OphyAI: build three stakeholder stories in Interview Practice and get per-answer feedback on whether the result is specific.

How to Practice These

Split preparation by round: work probability problems by hand, practice one full experiment design until the order is automatic, write cohort and funnel SQL without autocomplete, prepare one model you can defend for 20 minutes, and build five behavioral stories. Our data analyst interview questions cover the reporting side and Python interview questions the coding screen; data engineer interview questions cover the pipelines behind your data.

• OphyAI Interview Practice runs voice or text mock interviews in behavioral, technical and case formats, with follow-up questions, a transcript and per-answer feedback, so you can rehearse experiment designs and model walkthroughs before the real loop.
• OphyAI Interview Copilot gives real-time answer suggestions during live Zoom, Microsoft Teams and Google Meet interviews.

Start practicing →

Frequently Asked Questions

What are the most common data scientist interview questions?

The most common are explaining a p-value or confidence interval, a probability puzzle, designing and sizing an A/B test, interpreting a surprising test result, a SQL query on cohorts or funnels, a modeling walkthrough with evaluation metrics, a product metric case, and behavioral questions about stakeholders and mistakes. Product analytics roles weight experiments and SQL; machine learning roles weight modeling and coding.

How is a data scientist interview different from a data analyst interview?

Data scientist loops go deeper on statistics, experiment design and machine learning, and usually include a coding or modeling round. Analyst interviews weigh SQL, dashboards, reporting and stakeholder communication more heavily. The boundary is blurry, so read the posting and ask the recruiter what each round covers.

Do data scientist interviews include a take-home assignment?

Some do. Capital One’s own careers site says data science candidates complete a take-home Data Science Challenge before Power Day, covering data and engineering analytics, written communication and predictive modeling. Other employers use live coding or case rounds instead. Ask the recruiter about the format, time limit and whether you will present your work afterward.

How much statistics do I need for a data scientist interview?

Expect hypothesis testing, p-values and confidence intervals, power and sample size, Bayes’ rule, common distributions, regression basics, and experiment pitfalls such as peeking, multiple comparisons and sample ratio mismatch. Inference-focused or research roles may add causal inference methods. You rarely need proofs, but you do need to explain why a method fits the problem.

How much do data scientists make in the US?

The BLS reports a median of $120,230 a year for data scientists in May 2025, with the job projected to grow 35% from 2025 to 2035. Pay at large tech companies often adds stock, but those figures are candidate-reported and vary by level and city.

How does OphyAI work in a data scientist interview?

Rehearse the experiment, metric and modeling answers aloud in Interview Practice, which asks follow-up questions and returns a transcript and per-answer feedback. On live Zoom, Microsoft Teams or Google Meet rounds, Interview Copilot gives real-time answer suggestions and a live transcript. For live SQL or coding rounds, Coding Interview (Coding Copilot in the desktop app) is on the Premium plan.

How long does the data scientist interview process take?

It varies by employer. Pinterest publishes its machine learning and data science new grad path: an online assessment, a phone screen, a hiring manager review, 3-4 technical interviews and a manager call. Third-party guides report about 4-6 weeks for Meta’s data science process. Ask your recruiter for target dates, especially if you hold another offer.

Related Guides

Company guides where data science interviews show up:

• Meta interview guide
• Pinterest interview guide
• Airbnb interview guide
• Netflix interview guide
• Spotify interview guide
• Capital One interview guide and Capital One Power Day case interview
• LinkedIn interview guide
• DoorDash interview guide and Uber interview guide: marketplace experiments
• Google interview guide and Amazon interview guide

Related question guides:

• Machine learning interview questions: calibration, boosting, attention, ML system design and drift for ML engineer loops
• Data analyst interview questions and answers
• SQL interview questions
• Python interview questions
• Product manager interview questions
• Behavioral interview questions and answers
• Questions to ask the interviewer

Sources and verification notes

Sources checked October 2026. Questions and sample answers are original templates, not published by any employer; projects and numbers in them belong to a fictional candidate. Calculations were worked by hand; the sample size rule (n = 16 x variance / effect squared per group, 5% two-sided, 80% power) is a standard textbook approximation, not from a single fetched page. The SQL is PostgreSQL-style, reviewed by hand, not run.

• US Bureau of Labor Statistics, Occupational Outlook Handbook: Data Scientists: May 2025 median pay, 2025 jobs, 2025-2035 projected growth, annual openings, typical education and duties (page modified August 27, 2026).
• Capital One Careers, What to expect during your Capital One interview: the take-home Data Science Challenge before Power Day and its focus areas (no publication date shown).
• Pinterest Careers, Interview process: machine learning and data science interns and new grads: published interview steps (dated April 1, 2024).
• Meta Careers, hiring process: recruiter conversation, screening and full loop. The page does not describe data science rounds.
• DataLemur, Meta data science interview guide (updated March 2025): third-party and candidate-reported; the 4-6 week timeline and the analytical execution, analytical reasoning, technical skills and behavioral rounds.
• Spotify Engineering, Choosing a sequential testing framework (March 2023): Spotify’s experimentation platform uses group sequential tests; why peeking inflates false positives.
• American Statistical Association, Statement on p-values (2016): the six principles, including that a p-value does not measure the probability that a hypothesis is true or the size of an effect.
• scikit-learn, Common pitfalls: data leakage and Cross-validation: leakage definition, never fitting on test data, and time-series splits.