Computing Canvas Page-View Engagement Scores per Student

This guide computes a single per-student engagement score for a Canvas course from the platform’s page-view and participation analytics. The output is one 0–1 number per learner per week, baselined against the course cohort so it is comparable across sections, with every student identifier tokenized before it leaves memory. It is the single-vendor instance of the Engagement Metric Normalization logic, and it feeds directly into the at-risk scoring model under Engagement Analytics & Reporting Automation. The task is precise: raw Canvas analytics in, one comparable score per student out.

Prerequisites

Confirm each of these before running the script — it assumes all of them.

• Python 3.10+ (the script uses str | None unions and dataclass syntax).
• pandas==2.2.*, numpy, and requests==2.32.* installed in the virtualenv.
• A Canvas access token with the url:GET|/api/v1/courses/:course_id/analytics/... scope, exported as CANVAS_TOKEN. For unattended jobs, mint it via automating Canvas API token refresh in Python rather than a long-lived paste.
• The numeric course_id, with a teacher or admin enrollment (analytics endpoints require grading rights on the course).
• A tokenization salt in ENGAGEMENT_SALT, held in a secrets manager and rotated on schedule — never hard-coded. This is the salt from implementing salted SHA-256 tokenization for student ids.
• Upstream data shape: GET /api/v1/courses/:id/analytics/student_summaries returns one object per student with id (the Canvas user_id), page_views, and participations (an integer count of graded interactions).

Step-by-step implementation

1. Page the student_summaries analytics endpoint with the Link header. Canvas paginates analytics with RFC 5988 Link headers and rel="next"; stopping when a page looks short silently drops students, so follow next until it is gone — the same contract as the pagination strategies for bulk exports.

2. Extract page_views and participations per student, keeping them separate. They measure different things — passive viewing versus graded interaction — so you weight them differently in step four rather than summing them raw.

3. Tokenize the Canvas user_id immediately, before any aggregation. A salted SHA-256 turns the raw id into a stable stu_… token and the raw id is dropped, so nothing downstream — including any log line — handles a direct identifier, per the FERPA-Compliant PII Handling rules.

4. Weight and sum into one activity scalar. A participation reflects more genuine effort than a page view, so multiply each by its taxonomy weight before summing. This collapses two counts into the single weighted_activity the normalization logic expects.

5. Z-score within the cohort and squash to 0–1. Baseline each student’s scalar against the course mean and standard deviation, then map it through a logistic so the score is a bounded, comparable index where 0.5 is exactly the cohort average — the transform defined in Engagement Metric Normalization.

Complete runnable code block

import hashlib
import logging
import os

import numpy as np
import pandas as pd
import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)s | %(message)s")
logger = logging.getLogger("canvas_engagement")

# Effort weights: a graded participation counts for more than a passive page view.
W_PAGE_VIEW = 1.0
W_PARTICIPATION = 4.0
NEUTRAL_INDEX = 0.5  # fallback when the cohort has zero variance (cold start)


def tokenize(raw_id: str, salt: str) -> str:
    """FERPA-safe, deterministic, non-reversible learner token."""
    digest = hashlib.sha256(f"{salt}:{raw_id}".encode("utf-8")).hexdigest()
    return f"stu_{digest[:32]}"


def _next_url(link_header: str | None) -> str | None:
    """Extract the rel="next" URL from a Canvas RFC 5988 Link header."""
    if not link_header:
        return None
    for part in link_header.split(","):
        if 'rel="next"' in part:
            return part.split(";")[0].strip("<> ")
    return None


def fetch_student_summaries(api_base: str, token: str, course_id: int) -> list[dict]:
    """Page the analytics student_summaries endpoint until Link rel=next is gone."""
    url = f"{api_base}/api/v1/courses/{course_id}/analytics/student_summaries"
    headers = {"Authorization": f"Bearer {token}"}
    params: dict | None = {"per_page": 100}
    rows: list[dict] = []
    while url:
        resp = requests.get(url, headers=headers, params=params, timeout=30)
        # Canvas throttles with 403 (Rate Limit Exceeded), not only 429.
        if resp.status_code in (403, 429):
            raise RuntimeError(f"Throttled ({resp.status_code}); add backoff before retry.")
        resp.raise_for_status()
        rows.extend(resp.json())
        url = _next_url(resp.headers.get("Link"))
        params = None  # the next URL already carries the query string
    logger.info("Fetched %d student summaries for course %d", len(rows), course_id)
    return rows


def score_engagement(summaries: list[dict], salt: str) -> pd.DataFrame:
    """Tokenize, weight, z-score within the cohort, and squash to a 0-1 index."""
    if not summaries:
        return pd.DataFrame(columns=["learner_token", "weighted_activity",
                                     "z", "engagement_index"])
    df = pd.DataFrame(summaries)
    # 3 · Tokenize immediately, then drop the raw Canvas user id.
    df["learner_token"] = df["id"].astype(str).map(lambda x: tokenize(x, salt))
    df = df.drop(columns=["id"])
    # 4 · Weight the two activity signals and sum to one scalar.
    df["page_views"] = pd.to_numeric(df.get("page_views"), errors="coerce").fillna(0)
    df["participations"] = pd.to_numeric(df.get("participations"), errors="coerce").fillna(0)
    df["weighted_activity"] = (
        W_PAGE_VIEW * df["page_views"] + W_PARTICIPATION * df["participations"]
    )
    # 5 · Baseline within the course cohort (z-score), then logistic squash.
    mu = df["weighted_activity"].mean()
    sigma = df["weighted_activity"].std(ddof=0)
    if sigma and sigma > 0:
        df["z"] = (df["weighted_activity"] - mu) / sigma
        df["engagement_index"] = (1.0 / (1.0 + np.exp(-df["z"]))).round(4)
    else:  # zero-variance cohort: everyone identical -> neutral
        df["z"] = np.nan
        df["engagement_index"] = NEUTRAL_INDEX
    keep = ["learner_token", "page_views", "participations",
            "weighted_activity", "z", "engagement_index"]
    return df[keep].sort_values("engagement_index", ascending=False)


if __name__ == "__main__":
    frame = score_engagement(
        fetch_student_summaries(
            api_base="https://canvas.instructure.com",
            token=os.environ["CANVAS_TOKEN"],
            course_id=int(os.environ["COURSE_ID"]),
        ),
        salt=os.environ["ENGAGEMENT_SALT"],
    )
    logger.info("Scored %d students", len(frame))
    print(frame.head(10).to_string(index=False))

Verification and output validation

• One row per student, tokenized. assert frame["learner_token"].str.startswith("stu_").all() and assert "id" not in frame.columns — no raw Canvas id survived.
• Index is bounded. assert frame["engagement_index"].between(0, 1).all() — a value outside [0, 1] means the logistic squash was skipped.
• Mean maps to the midpoint. In a normal cohort, frame.loc[frame["z"].abs() < 1e-6, "engagement_index"] should be about 0.5; a student exactly at the cohort mean scores the midpoint.
• Ordering is sane. The student with the most weighted activity has the highest index: assert frame.iloc[0]["engagement_index"] >= frame.iloc[-1]["engagement_index"].
• Cold-start fallback. For a single-student or all-identical cohort, assert (frame["engagement_index"] == 0.5).all() — the zero-variance branch fired instead of dividing by zero.

Troubleshooting

• KeyError: 'participations' on some students. Canvas omits participations for students with zero graded interaction rather than returning 0. The df.get("participations") plus fillna(0) handles it; if you index the column directly, guard with .get() first.
• 403 Forbidden (Rate Limit Exceeded) mid-pull. Canvas throttles analytics reads with 403, not only 429 — they are cost-expensive. The script raises a distinct RuntimeError; layer Handling Canvas API Rate Limits on top so the run paces instead of aborting.
• Every index is exactly 0.5. The cohort has zero variance — usually because you ran against a one-student sandbox or a course where analytics has not populated. This is the cold-start fallback working; validate against a real multi-student course.
• Indices look inverted (active students low). You summed page_views and participations with equal weight, letting a passive viewer outscore an active contributor. Keep W_PARTICIPATION above W_PAGE_VIEW so genuine interaction dominates.
• 401 Unauthorized partway through pagination. A short-lived token expired mid-export. Refresh proactively via automating Canvas API token refresh in Python and re-fetch a valid token at the top of each page.
• Scores swing week to week for the same student. You are comparing raw indices across cohorts of different size, where a small section’s $\sigma$ is unstable. Apply a minimum-cohort threshold and read the caveats in Engagement Metric Normalization.

Related

• Engagement Metric Normalization — the parent guide defining the weight-baseline-squash logic this script implements for one vendor.
• Normalizing LMS Login Frequency Across Platforms — the sibling task extending this idea across Canvas, Moodle, and Blackboard.
• Building an At-Risk Student Flag in Python — how this engagement index becomes one input to an early-warning flag.
• Handling Canvas API Rate Limits — pacing the cost-expensive analytics reads this script depends on.

Part of: Engagement Metric Normalization