How to use these tools as a coach

Keep your own protocol

Most coaches already run a norming or calibration protocol they trust. Keep it. The scoring sheet adds one thing to whatever you run: the two numbers and the log at the end. If your protocol reveals scores before independent scoring, move the reveal after step 2, since the numbers only mean something when each scorer commits alone.

Introduce it without threat

Say the sentence from page 1 out loud in the first session: the number describes the scale wording and the task, and it says nothing about any one scorer. Then keep the promise. Fill the grid yourself, leave names off the rows, and copy only the totals into the log. A team that has seen the number used in an evaluation will stop producing an honest one.

Use the log as the coaching agenda

The “one decision” column is the record of what the team changed each month. Read it before each visit. If the same descriptor gets rewritten three sessions running, the scale needs a structural change, and rewriting that descriptor again will not move the number. If exact agreement rises while adjacent stays flat, the team has a two level disagreement it has not talked about. Go find the O cells.

Watch for the usual traps

• Small grids. Three scorers and four samples give 12 scores, so one disagreement moves exact agreement by eight points. Read the trend across sessions. A single number moves too easily.
• Ties. When the mode ties, the facilitator picks and stars the cell. Two stars in one grid means the scale’s middle levels are not distinct yet.
• Mixed targets. Only compare sessions on the same target and scale. A number from a writing rubric says nothing about a math scale.
• Anchor drift. High agreement can mean the team has agreed to read the standard generously. Once a year, score an anchor sample against the standard itself.

Bring the credit rule to the grading conversation

Schools moving to retakes and revisions argue about deadlines first. The template moves the argument to the question that changes grades: how support before a resubmission counts. Run section 3 as a card sort with the whole department. Expect the “hint during the attempt” and “second attempt on the same task” rows to split the room. Section 8 settles the argument with the school’s own data at the end of the semester.

What to send back to a network

Aggregate only. A network can learn from “eleven schools, median exact agreement 64 percent in October and 72 percent in March” without a single school being named. Never forward a scoring grid, and never forward scorer identities, which the sheet does not record anyway.

A cadence that has worked

One session per target per month for one semester, the credit rule written by the second session, the section 8 check in the last week. Then decide whether to add a second target.