criterion performance measurements

overview

want to understand this report?

Y2015/Day 1/simple

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.2103735158683023e-7 1.2229128719760622e-7 1.2361309985746027e-7
Standard deviation 3.548271047431596e-9 4.216091741964892e-9 5.198169651800269e-9

Outlying measurements have severe (0.5280297384399185%) effect on estimated standard deviation.

Y2015/Day 1/large

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.129371364575472e-7 3.163013895333433e-7 3.1973069401640093e-7
Standard deviation 9.756091825432612e-9 1.1742377287292224e-8 1.4781321809652265e-8

Outlying measurements have severe (0.5464807332064489%) effect on estimated standard deviation.

Y2015/Day 1/huge

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.561900572639469e-6 4.6203898949721455e-6 4.680922212290819e-6
Standard deviation 1.6352810708515063e-7 1.9724790268086704e-7 2.445435253715343e-7

Outlying measurements have severe (0.5461150320707798%) effect on estimated standard deviation.

Y2015/Day 22/simple

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.776152907571032e-5 2.8053540665241866e-5 2.8447569674202658e-5
Standard deviation 8.739815869205005e-7 1.1160223600355853e-6 1.5455813340671673e-6

Outlying measurements have moderate (0.4533752419012217%) effect on estimated standard deviation.

Y2015/Day 22/alternate

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.8826669282116288e-5 2.9200154955447082e-5 2.9639305485173323e-5
Standard deviation 1.0897954637809988e-6 1.3570091364768063e-6 1.696214479697152e-6

Outlying measurements have severe (0.5339231165885626%) effect on estimated standard deviation.

Y2015/Day 24/idealEntanglement/small

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.949832656670251e-6 1.0154269487993754e-5 1.0527638356001298e-5
Standard deviation 5.129148694376222e-7 9.068985560540332e-7 1.5127174970252797e-6

Outlying measurements have severe (0.8330062145764585%) effect on estimated standard deviation.

Y2015/Day 24/idealEntanglement/large

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.062715035374857e-5 6.189765590381671e-5 6.644753736818686e-5
Standard deviation 2.1201786629934535e-6 6.707586686398517e-6 1.469773193136266e-5

Outlying measurements have severe (0.8493844651544947%) effect on estimated standard deviation.

Y2015/Day 24/idealEntanglementOptimized/small

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1482362270134189e-5 1.1643108934669458e-5 1.1899490654517313e-5
Standard deviation 4.191079320234109e-7 6.623653691551125e-7 1.1291935302492724e-6

Outlying measurements have severe (0.6632738444655104%) effect on estimated standard deviation.

Y2015/Day 24/idealEntanglementOptimized/large

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1872505030303104e-4 1.2013899042288835e-4 1.2200831322968541e-4
Standard deviation 4.46523748462392e-6 5.50253376298412e-6 7.067614217436239e-6

Outlying measurements have moderate (0.4711102822276888%) effect on estimated standard deviation.

Y2016/Day 1/blockDistance/simple

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.191145772144765e-6 4.406453799553005e-6 4.891888877793722e-6
Standard deviation 6.453342852853336e-7 1.03794939696902e-6 1.7481603081790766e-6

Outlying measurements have severe (0.9765419431544073%) effect on estimated standard deviation.

Y2016/Day 1/blockDistance/larger

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.946259241801556e-6 9.421172964757427e-6 9.988826407185273e-6
Standard deviation 1.2093687145320356e-6 1.706808960973403e-6 2.408016577633739e-6

Outlying measurements have severe (0.9549301678641865%) effect on estimated standard deviation.

Y2016/Day 1/visitedTwice/simple

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.2980814619407821e-5 1.3812465251125846e-5 1.6434134080096407e-5
Standard deviation 1.3102344733410248e-6 4.211215792798075e-6 9.19123354285011e-6

Outlying measurements have severe (0.9863046608519244%) effect on estimated standard deviation.

Y2016/Day 2/bathroomCode/part 1

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1041779392976851e-6 1.1330223589711345e-6 1.164719919000394e-6
Standard deviation 8.0463874091027e-8 9.800878636978336e-8 1.3104667845372738e-7

Outlying measurements have severe (0.8538330822368547%) effect on estimated standard deviation.

Y2016/Day 2/bathroomCode/part 2

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1702004042945898e-6 1.2011450098404906e-6 1.2490642170860314e-6
Standard deviation 9.999959148260399e-8 1.3340266638209468e-7 2.0765726063819928e-7

Outlying measurements have severe (0.9083799547362655%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.