[AliAlfaifi]

Why

In 0.13.9 the first call of the bulk progress feed (GET /api/fx/learners/v1/learner_courses/, no username) ran a live COUNT over the caller's whole scope. For a platform-wide caller that takes minutes: Cloudflare cuts the request at about 100 s while MySQL keeps counting. On 2026-10-22, six such counts were stacked on the /api/fx read replica (110–401 s each) and had to be killed.

What

The response does not change: count and page stay in every response.

• count is now the sum of CourseStat.progress_count over the caller's courses, which takes milliseconds (get_learners_progress_count).
• With modified_since, count is null, because a total of changed rows can't be kept in advance.
• sync_course_stats (the existing stats job) fills the new progress_count. It reads the course-level completion aggregates, with course keys in batches of 500, so the (course_key, aggregation_name, …) index answers each batch as a range read.

Measured on the prod read replica (read-only), 2026-10-22/24:

Sync query	Time
exact: each enrollment checked against its user and progress	> 300 s (cut)
progress index without course keys (scans all 416M entries)	> 300 s (cut)
this PR: progress index, course keys in batches of 500	13.3 s (4,165 courses, 9 batches, slowest 7.6 s)

Trade-offs (documented and tested):

• count is as fresh as the last sync.
• It counts learners who unenrolled or were deactivated after starting, so it can be slightly above the rows walked.

Release notes

• Migration 0013_coursestat_progress_count (adds the column), so the deploy must run init (skip_init=false).
  • ⚠️ The unreleased 0.14.0 branch also has a 0013_backfill_view_allow_write. When it rebases onto this, rename it to 0014 and make it depend on 0013_coursestat_progress_count.
• Run ./manage.py lms sync_course_stats --async once right after the deploy. Until then every count reads 0.
• Schedule: a companion nassau PR moves CRONJOB_SCHEDULE_SYNC_COURSE_STATS from nightly to hourly at :20, after the hourly aggregator run.
• __version__ 0.13.10.

Tests

tox -e py311-teak,py311-redwood,py311-sumac,quality: 100% coverage in each release, quality clean. New tests cover:

• progress_count: course-level only, above 0, counted across batches, courses with progress but no certificates;
• the documented overcount;
• the count follows the caller's scope;
• the count is stale until the next sync;
• 0 before the first sync, null with modified_since.

🤖 Generated with Claude Code