Skip to content
Nitesh Tiwari

Edfora · EdTech · Professional experience

Adaptive Assignment Engine

Why a fixed practice sequence lost learners, and what changed when question difficulty was matched to each learner’s ability.

Outcome
Assignment completion: 18% → 45%
Focus
Personalization3PL IRTLearning
Role
Senior Product Manager · Edfora · 2023–2026
Evidence
Documented outcome

Edfora’s learning and engagement products reached 100K+ learners overall; that figure is not specific to this engine. This case study covers product reasoning and outcomes. Proprietary implementation details, internal data and confidential employer information are left out.

The 30-second version

Read the full story

  1. 1Problem

    A fixed practice sequence gave every learner the same next question: too hard for some, too easy for others. Assignment completion was 18%.

  2. 2Approach

    Three options weighed: learner choice, rule-based difficulty bands, or ability estimation. Chosen: a 3PL IRT-based engine that estimates each learner’s ability (θ) and selects question difficulty to match.

  3. 3Outcome

    Across a 2-year academic-cycle dataset: 18% completion under the static path, 45% after the adaptive system was introduced (+27 pts).

01

Problem

Problem

Learners had the material and still stopped.

Low assignment completion was a key driver of learners dropping off. Every learner got the same fixed sequence, and one sequence can’t be the right difficulty for learners at different levels. A fit problem, not a content problem.

Illustrative model

  1. Too hard

    Frustration and disengagement

  2. Right fit

    Learner keeps progressing

    Adaptation aims here

  3. Too easy

    Low cognitive value

A uniform sequence places every learner at the same difficulty, so some land left of the fit and some land right of it.
02

Decision

Options

Three ways to fix difficulty fit.

Let learners choose, move them between difficulty bands with rules, or estimate each learner’s ability and match questions to it. The third was chosen.

Product reasoning

  1. A

    Let learners choose their difficulty

    Works because

    Simple to build, and gives learners control.

    Fails because

    The learners who most need an easier path are the least able to judge it, and it adds a decision before they start.

  2. B

    Move learners between bands with rules

    Works because

    Easy to build and easy to explain to teachers.

    Fails because

    Treats every question as equally informative, so a lucky guess counts the same as real mastery.

  3. C

    Estimate ability and match calibrated questions

    Direction taken

    Works because

    Weighs each answer by how much it actually reveals, and discounts guesses.

    Costs

    Needs calibrated question parameters, and is harder to explain than a sequence.

Match the question to the learner, not the learner to the sequence.

Hypothesis

Product reasoning
If
each learner gets questions matched to their current ability instead of a fixed sequence
Then
fewer will hit a wall or coast, and more will finish the assignment
Measured by
Assignment completion (primary), with practice drop-off as the second signal

My role

Senior PM, Core Practice & Learning Experience

One PM (me), one APM, one product designer, 5–7 engineers and 2–3 academic leads. I owned the strategy, roadmap, problem analysis, PRD and adaptive product logic, and post-launch tracking.

03

How it works

Mechanism

3PL IRT: learner ability on one side, question parameters on the other.

The engine estimates each learner’s ability (θ) per concept from their performance history. Every question carries a difficulty, discrimination and guessing parameter. The engine selects questions targeted at the learner’s current ability, then updates θ after each response.

Matching a question to a learner with 3PL IRT

Illustrative model
Candidate questions for a learner whose estimated ability sits in the middle of the scale, with the uncertainty band shown around it
QuestionVerdict
Q1Too easy, and tells us littleDiscrimination low · guessing low
Q2Right level, but a correct answer could be a guessDiscrimination high · guessing high
Q3Served nextDiscrimination high · guessing low
Q4Too hard for nowDiscrimination high · guessing low
After each answer θ is updated, which changes the next choice. Positions are illustrative; real parameters, thresholds and selection rules are not shown.
04

Outcome

Outcome

Assignment completion: 18% → 45%.

Across a 2-year academic-cycle dataset, assignment completion was 18% under the static learning path and 45% after the adaptive system was introduced: +27 percentage points.

Concurrent product changes in that period aren’t on record, so the increase isn’t attributed to the adaptive system alone.

Assignment completion

Documented outcome

18% → 45%

+27 ptsEdfora · 2-year academic-cycle dataset · before vs. after

Practice drop-offs also reduced.

05

Learning

Trade-off

Product reasoning

What ability-based matching costs.

It is harder to explain than a fixed sequence, it depends on well-calibrated questions, and a new learner’s first questions carry the most uncertainty.

Learning

When completion drops, check the fit before adding content.

Personalize only where it clearly improves the job. Everywhere else, a stable default wins.