supremecourt.report · methods note

Predicting the Probability of Certiorari

How this site estimates the chance that a petition will be granted, and how those estimates are checked against what the Court actually did.

What the number means

Every paid petition on this site carries a percentage: the estimated chance that the Court will grant it and hear the case. It works like a rain forecast. A “17%” does not say the petition scores 17 out of 100, and it does not say the petition will be granted. It says that, of a hundred petitions that look like this one on the docket, about seventeen have been granted in the past. Most petitions are long shots: across the eight Terms the models learned from, 4.1% of paid petitions were granted, so a petition rated near that figure is simply average.

Three estimates for three moments

A petition’s prospects change as its docket develops, so the site carries three related estimates rather than one.

• At filing. The baseline estimate uses only what is known on the day a petition is docketed: who is asking and whom they are asking against, which court ruled below, how quickly the case came up, whether the petition reports a dissent below or a split among the lower courts, how long the petition is (as its filer certified) and how many dissents it cites, and the lawyers’ record at the Court. This is the number on the daily petition dashboards and on each case page.
• At conference. The enhanced estimate keeps the dissent and split cues and adds what happened next: whether the other side filed a response or waived it, whether the Court asked for one, whether outside groups filed briefs in support, and whether the Justices have already considered the petition once and carried it over to another conference (a “relist”). This is the number on the conference reports.
• The other way to win. Some petitions are neither granted nor denied. When the Court has just decided a related case, it often sends similar petitions back to the lower court to reconsider, without briefing or argument. Lawyers call this a GVR, for granted, vacated and remanded. The GVR estimate is the chance of that outcome. It sits beside the grant estimate because a petition being held for a related case can look, on the docket, very much like one about to be granted.

Petitions filed without the fee, by people who cannot afford it, are not scored. About one in a thousand of them is granted, and a model built for paid petitions would be wrong about them in both directions.

What the models learned from

Eight Terms of the Court’s own docket records, from the Term that began in October 2017 through the one that began in October 2024: 11,814 paid petitions, of which 498 were granted and heard. A petition counts as a success only if the Court took the case in full. Petitions sent back with a GVR, dismissed, or still waiting are left out of the lesson, because they are neither a yes nor a no to the question the model is asking.

One rule matters more than any other: the models may see only what was on the docket before the Court acted. A response filed the week after a grant says nothing useful about predicting that grant, and letting it in would make the model look far better than it is. Every docket signal is measured strictly before the decision date.

How the estimate is made

The method is the plainest one that works: a scoring formula. Each fact about a petition adds or subtracts points (the United States as petitioner adds a great many; a petitioner with no lawyer subtracts a great many), the points are totalled, and the total is converted into a probability. Statisticians call this logistic regression. Nobody writes the points by hand: the formula is fitted to the eight Terms of outcomes so that each fact carries the weight it actually had.

The conversion from points to a percentage is then checked and, where needed, adjusted, a step called calibration, so that when the model says 17%, about 17 in 100 such petitions really were granted.

The checking is done honestly. Fitting a model and then grading it on the same petitions would be like letting a student mark their own homework with the answers open. Instead, each Term is hidden in turn: the model is fitted on the other seven Terms, asked to predict the hidden one, and graded against what actually happened. Every figure in the next section comes from those hidden-Term predictions, never from petitions the model had already seen. A stricter version of the check fits only on Terms earlier than the one being predicted, as a forecaster working in real time would have to. It scores slightly lower, by 0.006 to 0.025 on the AUC scale explained below depending on the estimate. That is the honest cost of predicting forward in time, and the figures below are a little more generous than it.

How well it works

Base rate is how often the outcome happened at all: the share of petitions in each model’s data that were granted, or for the GVR model sent back. It is the number a model has to beat.

AUC answers a simple question. Pick one granted petition and one denied petition at random; how often does the model rate the granted one higher? A coin flip scores 0.5 and a perfect model scores 1. The baseline scores 0.876, so it puts the granted petition first about 88% of the time.

Average precision asks how concentrated the grants are near the top of the model’s ranking. A random ranking scores the base rate itself, 4.14%. The baseline scores 0.292, about 7.1 times better than chance.

Brier score is the average error of the percentages themselves, so lower is better. The comparison is with the laziest possible forecast, which gives every petition the base rate. That forecast scores 0.0397 for the baseline, 0.0720 for the enhanced model and 0.0516 for the GVR model, and all three models beat it.

Plain accuracy is not reported because it would mislead. A model that said “denied” to every petition would be right 95.9% of the time and would be worthless.

These figures test the model on past Terms. Its live record, scored on forecasts written down before each conference, is on the Forecast Scorecard; see “Keeping score” below.

What moves the estimate

• The United States as petitioner. When the Solicitor General asks, the Court usually listens: roughly 43% of those petitions are granted, against about 3.7% of everyone else’s. Nothing else on the docket comes close.
• Relists. When the Justices take up a petition at conference and, instead of deciding it, carry it over to the next conference, that is a relist. One relist is a strong sign: about 20% of those petitions are granted, against roughly 1% of petitions never relisted. Two is stronger still, about 44%. But five or more usually means the petition is being held for another case, and the grant rate falls back to about 19%. Because the pattern rises and then falls, the model treats each count separately instead of assuming that more is always better.
• A dissent below, or a split among the lower courts. Read from the petition itself. The Court’s Rule 10 names disagreement among the lower courts as the main reason it takes cases, and petitions that report one are granted more often.
• The heft of the petition. How long it is, taken from the word count every booklet petition must certify under Rule 33.1(h) (a separate certificate for most filers, a line in the proof of service for the Solicitor General), read against the 9,000-word limit; and how many dissents it cites, read from the PDF. Neither is a reason to grant, but both track how much work went into the petition. A petition under 3,000 words is granted well under 1% of the time; one that cites more than ten dissents, about 16%. Both are taken in bands rather than as a straight line, and both sit beside counsel’s record, which they partly repeat.
• Counsel’s track record. How many petitions the lawyers had filed before this one and how many were granted, counted only up to the day this petition was docketed so that later successes cannot leak in.
• The court below. Petitions from the federal courts of appeals are granted far more often than petitions from state courts.

Does 17% really mean 17%?

Within the limits of the data, yes. Sort the petitions into ten groups by the model’s estimate and compare each group’s average estimate with the share actually granted. In the riskiest tenth, the enhanced model predicted 39.0% and 36.9% were in fact granted; the baseline predicted 23.1% and 23.2% were granted. The lower groups track just as closely, as the chart above shows.

Keeping score

Every figure above was computed after the outcomes were known, on Terms the model was not fitted to. That is an honest test, but it is still a test set up after the fact. So the site also keeps a record it cannot revise. Before each conference, the nightly run writes down a forecast for every paid petition on the list and never edits it afterwards. Nothing is written for a conference that has already happened.

Each petition gets two forecasts at each conference. The first forecast is written the first night the petition appears on an upcoming conference’s list, often weeks ahead. The final forecast is made again every night after that and frozen at noon UTC on the day of the conference, before the Justices meet. It reflects everything the docket showed by then: the response, any relists, an invitation to the Solicitor General.

The Forecast Scorecard holds both to the orders the Court then issued, conference by conference: how many grants the forecasts expected and how many came, where each grant ranked among the petitions at its conference, and the same AUC and calibration checks as above. Its numbers are less settled than these, because the Court grants only a few dozen petitions a Term, but they are numbers the model could not have seen.

What it cannot do

• Grants are rare: only 498 in eight Terms. Rare events are hard to learn from, and the estimate for an unusual petition is less certain than the estimate for a typical one.
• The last Term was unfinished. The most recent Term in the data was still in progress when the data was taken, so petitions filed late in it had not been decided. That makes it look like a harder Term than it was.
• Some petitions cannot be read. About 9% have no readable PDF. For those the model assumes no dissent below, no split and no dissents cited, which if anything understates their chances. A petition whose word count could not be read from the docket is scored as one of unknown length.
• Names are matched by pattern. Deciding what kind of party is asking (a business, a state, the federal government) and recognising the same lawyer across petitions is done by matching names. It is good, not perfect.
• The model knows nothing about the law. It has never read the question presented. It sees only the shape of the docket, which is why a strong petition on a subject the Court has been avoiding can score low, and an ordinary one from the Solicitor General can score high.

Coefficients

This section is for readers who want to check the arithmetic. It lists every term in every model exactly as deployed, read from the model files themselves. For everyone else the sections above are the whole story, and these tables can be skipped.

Each row is one fact the model can see. β is the points that fact adds to a petition’s score (a negative number subtracts), and the odds ratio, e^β, restates the same thing as a multiplier on the odds of the outcome with everything else held equal: 2.0 means the odds double, 0.5 means they halve. SE is the uncertainty in the estimate, z is the estimate divided by that uncertainty, and p is the chance of seeing an effect at least this large if the true effect were zero. A • marks p below 0.05. It is a reading aid, not a finding: this is eight Terms of observational data, the 118 tests reported here are not corrected for their number, and a coefficient measures association, not cause.