Hugging Face's logo

dizyann
/
crivo-web

Transformers.js
ONNX
English
custom
modernbert
typed-decisions
quantized
int8
browser
wasm
webgpu
Model card Files
xet
 Community
dizyann commited on
Commit
b01a4ff
·
1 Parent(s): bd8c5ee

docs: publish browser benchmarks for every model variant

Browse files
Files changed (9) hide show
  1. README.md +46 -0
  2. models/150m/README.md +4 -1
  3. models/17m/README.md +4 -1
  4. models/1b/README.md +4 -1
  5. models/32m/README.md +4 -1
  6. models/400m/README.md +4 -1
  7. models/68m/README.md +4 -1
  8. scripts/benchmark/browser.json +2351 -0
  9. scripts/benchmark/playground.js +2 -1
README.md CHANGED
@@ -86,6 +86,52 @@ The only runtime export remains `decide`; `ModelSize` and `ModelOptions` are typ
86
  An injected `decisionRuntime` takes precedence over `model` options. When composing
87
  through a Layer, provide `model: {}` alongside a runtime provider if using defaults.
88
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
  ### Download sizes and conversion checks
90
 
91
  Weights only; tokenizer and runtime downloads are additional. Memory use is higher.
 
86
  An injected `decisionRuntime` takes precedence over `model` options. When composing
87
  through a Layer, provide `model: {}` alongside a runtime provider if using defaults.
88
 
89
+ ### Measured in the browser playground
90
+
91
+ Measured with **`bun run browser`**, Apple M4 / 24 GiB, macOS 26.6.2,
92
+ headless Chromium 154, WebGPU via Metal, ONNX Runtime Web 1.30.0.
93
+ The page was not cross-origin isolated (WASM threading unavailable). This was a local
94
+ development session, without controlled power or thermal conditions.
95
+ The default `examples/basic` request contains **three questions in one batch**.
96
+ Each configuration had one initialization run, three more warmups, then 20 measured
97
+ runs with the same session. Times include tokenization, ONNX inference and answer mapping;
98
+ they exclude model setup and UI rendering. These are workload-specific measurements,
99
+ not latency promises or a rerun of Decision Index. Longer inputs and different devices
100
+ can change both latency and model rankings.
101
+
102
+ **Warm inference: median / p95, milliseconds per complete three-question request.**
103
+
104
+ | Model | CPU · WASM · INT8 | CPU · WASM · FP32 | GPU · WebGPU · FP32 |
105
+ | --- | ---: | ---: | ---: |
106
+ | 17M | 61.4 / 69.3 | 57.1 / 69.5 | 12.6 / 16.1 |
107
+ | 32M | 187.0 / 215.2 | 168.0 / 172.0 | 14.6 / 16.4 |
108
+ | 68M | 577.6 / 596.8 | 535.0 / 551.8 | 33.2 / 36.3 |
109
+ | 150M | 1466.2 / 1526.9 | 1302.0 / 1368.1 | 68.4 / 70.1 |
110
+ | 400M | 4406.5 / 4517.8 | 3903.4 / 4017.0 | 192.0 / 197.8 |
111
+ | 1B | 12442.0 / 13207.4 | Unsupported | 486.8 / 504.3 |
112
+
113
+ **Model setup from cached weights, seconds (one observation per configuration).**
114
+ This includes cached asset reads and session initialization, but not network downloads
115
+ or the first inference. Reusing a session makes this setup time zero on subsequent runs.
116
+
117
+ | Model | CPU · WASM · INT8 | CPU · WASM · FP32 | GPU · WebGPU · FP32 |
118
+ | --- | ---: | ---: | ---: |
119
+ | 17M | 0.36 | 0.19 | 0.21 |
120
+ | 32M | 0.22 | 0.26 | 0.30 |
121
+ | 68M | 0.42 | 0.67 | 0.75 |
122
+ | 150M | 0.62 | 1.33 | 1.58 |
123
+ | 400M | 1.27 | 4.73 | 4.82 |
124
+ | 1B | 3.72 | — | 6.31 |
125
+
126
+ INT8 reduces weight downloads; these measurements do not show a consistent speed advantage.
127
+ For interactive browser use, evaluate smaller variants on WASM or FP32 on WebGPU,
128
+ then validate decision quality on your own inputs. The 1B FP32/WASM combination is disabled
129
+ because it exceeded the tested memory budget; INT8/WebGPU is also unsupported.
130
+
131
+ [Raw samples and exact input](scripts/benchmark/browser.json) ·
132
+ [Reproduction procedure](scripts/benchmark/README.md) ·
133
+ [UI benchmark script](scripts/benchmark/playground.js).
134
+
135
  ### Download sizes and conversion checks
136
 
137
  Weights only; tokenizer and runtime downloads are additional. Memory use is higher.
models/150m/README.md CHANGED
@@ -11,7 +11,7 @@ Original architecture, training and weights by [tardellirs](https://huggingface.
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
- [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-150m-web)
15
 
16
  Source revision: `981e257e06537ddfbdc6e7215124875a42fd97c0`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-150m/blob/981e257e06537ddfbdc6e7215124875a42fd97c0/README.md)
@@ -29,3 +29,6 @@ FP32 maximum logit error against PyTorch on the three bundled fixtures:
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
 
 
 
 
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
+ [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-web)
15
 
16
  Source revision: `981e257e06537ddfbdc6e7215124875a42fd97c0`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-150m/blob/981e257e06537ddfbdc6e7215124875a42fd97c0/README.md)
 
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
32
+
33
+ This variant is served from `dizyann/crivo-web` under `models/150m/`.
34
+ Select it through the TypeScript API with `model: { size: "150m" }`.
models/17m/README.md CHANGED
@@ -11,7 +11,7 @@ Original architecture, training and weights by [tardellirs](https://huggingface.
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
- [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-150m-web)
15
 
16
  Source revision: `2b97905e8d35b0afbbdfa63d6dfc92d6c3450c25`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-17m/blob/2b97905e8d35b0afbbdfa63d6dfc92d6c3450c25/README.md)
@@ -29,3 +29,6 @@ FP32 maximum logit error against PyTorch on the three bundled fixtures:
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
 
 
 
 
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
+ [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-web)
15
 
16
  Source revision: `2b97905e8d35b0afbbdfa63d6dfc92d6c3450c25`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-17m/blob/2b97905e8d35b0afbbdfa63d6dfc92d6c3450c25/README.md)
 
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
32
+
33
+ This variant is served from `dizyann/crivo-web` under `models/17m/`.
34
+ Select it through the TypeScript API with `model: { size: "17m" }`.
models/1b/README.md CHANGED
@@ -11,7 +11,7 @@ Original architecture, training and weights by [tardellirs](https://huggingface.
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
- [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-150m-web)
15
 
16
  Source revision: `5dab4fb0ee59bbd08d0126027aff14ddf90466ee`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-1b/blob/5dab4fb0ee59bbd08d0126027aff14ddf90466ee/README.md)
@@ -29,3 +29,6 @@ FP32 maximum logit error against PyTorch on the three bundled fixtures:
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
 
 
 
 
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
+ [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-web)
15
 
16
  Source revision: `5dab4fb0ee59bbd08d0126027aff14ddf90466ee`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-1b/blob/5dab4fb0ee59bbd08d0126027aff14ddf90466ee/README.md)
 
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
32
+
33
+ This variant is served from `dizyann/crivo-web` under `models/1b/`.
34
+ Select it through the TypeScript API with `model: { size: "1b" }`.
models/32m/README.md CHANGED
@@ -11,7 +11,7 @@ Original architecture, training and weights by [tardellirs](https://huggingface.
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
- [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-150m-web)
15
 
16
  Source revision: `77d61fdb6421838320097d9bcb3e4e42a120323e`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-32m/blob/77d61fdb6421838320097d9bcb3e4e42a120323e/README.md)
@@ -29,3 +29,6 @@ FP32 maximum logit error against PyTorch on the three bundled fixtures:
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
 
 
 
 
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
+ [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-web)
15
 
16
  Source revision: `77d61fdb6421838320097d9bcb3e4e42a120323e`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-32m/blob/77d61fdb6421838320097d9bcb3e4e42a120323e/README.md)
 
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
32
+
33
+ This variant is served from `dizyann/crivo-web` under `models/32m/`.
34
+ Select it through the TypeScript API with `model: { size: "32m" }`.
models/400m/README.md CHANGED
@@ -11,7 +11,7 @@ Original architecture, training and weights by [tardellirs](https://huggingface.
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
- [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-150m-web)
15
 
16
  Source revision: `81f9cab8d7b3cc049b33033952850bb15ce654ae`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-400m/blob/81f9cab8d7b3cc049b33033952850bb15ce654ae/README.md)
@@ -29,3 +29,6 @@ FP32 maximum logit error against PyTorch on the three bundled fixtures:
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
 
 
 
 
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
+ [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-web)
15
 
16
  Source revision: `81f9cab8d7b3cc049b33033952850bb15ce654ae`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-400m/blob/81f9cab8d7b3cc049b33033952850bb15ce654ae/README.md)
 
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
32
+
33
+ This variant is served from `dizyann/crivo-web` under `models/400m/`.
34
+ Select it through the TypeScript API with `model: { size: "400m" }`.
models/68m/README.md CHANGED
@@ -11,7 +11,7 @@ Original architecture, training and weights by [tardellirs](https://huggingface.
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
- [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-150m-web)
15
 
16
  Source revision: `4519d10aa61562f1c74461ff84a45dcda0944e02`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-68m/blob/4519d10aa61562f1c74461ff84a45dcda0944e02/README.md)
@@ -29,3 +29,6 @@ FP32 maximum logit error against PyTorch on the three bundled fixtures:
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
 
 
 
 
11
  No additional training was performed. Weights retain the CC BY-NC 4.0 license.
12
 
13
  [Playground](https://huggingface.co/spaces/dizyann/crivo-demo) ·
14
+ [TypeScript library, CLI and model comparison](https://huggingface.co/dizyann/crivo-web)
15
 
16
  Source revision: `4519d10aa61562f1c74461ff84a45dcda0944e02`.
17
  See the [pinned original card](https://huggingface.co/tardellirs/crivo-68m/blob/4519d10aa61562f1c74461ff84a45dcda0944e02/README.md)
 
29
  benchmark; recalibrate thresholds on your own inputs and compare equivalent batches.
30
  See `parity.json` for fixtures, versions and provenance. Use the project lockfile:
31
  older native ONNX Runtime versions can produce materially different INT8 probabilities.
32
+
33
+ This variant is served from `dizyann/crivo-web` under `models/68m/`.
34
+ Select it through the TypeScript API with `model: { size: "68m" }`.
scripts/benchmark/browser.json ADDED
@@ -0,0 +1,2351 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "runs": [
3
+ {
4
+ "size": "17m",
5
+ "device": "cpu",
6
+ "dtype": "q8",
7
+ "samples": [
8
+ {
9
+ "loadMs": 0,
10
+ "inferenceMs": 61.30000019073486,
11
+ "disposeMs": 0,
12
+ "totalMs": 61.30000019073486
13
+ },
14
+ {
15
+ "loadMs": 0,
16
+ "inferenceMs": 60,
17
+ "disposeMs": 0,
18
+ "totalMs": 60
19
+ },
20
+ {
21
+ "loadMs": 0,
22
+ "inferenceMs": 60.5,
23
+ "disposeMs": 0,
24
+ "totalMs": 60.59999990463257
25
+ },
26
+ {
27
+ "loadMs": 0,
28
+ "inferenceMs": 60.39999961853027,
29
+ "disposeMs": 0,
30
+ "totalMs": 60.5
31
+ },
32
+ {
33
+ "loadMs": 0,
34
+ "inferenceMs": 60.799999713897705,
35
+ "disposeMs": 0,
36
+ "totalMs": 60.90000009536743
37
+ },
38
+ {
39
+ "loadMs": 0,
40
+ "inferenceMs": 69.30000019073486,
41
+ "disposeMs": 0,
42
+ "totalMs": 69.40000009536743
43
+ },
44
+ {
45
+ "loadMs": 0,
46
+ "inferenceMs": 75.69999980926514,
47
+ "disposeMs": 0,
48
+ "totalMs": 76.19999980926514
49
+ },
50
+ {
51
+ "loadMs": 0,
52
+ "inferenceMs": 68.2000002861023,
53
+ "disposeMs": 0,
54
+ "totalMs": 68.40000009536743
55
+ },
56
+ {
57
+ "loadMs": 0,
58
+ "inferenceMs": 68.80000019073486,
59
+ "disposeMs": 0,
60
+ "totalMs": 68.90000009536743
61
+ },
62
+ {
63
+ "loadMs": 0,
64
+ "inferenceMs": 65.09999990463257,
65
+ "disposeMs": 0,
66
+ "totalMs": 65.30000019073486
67
+ },
68
+ {
69
+ "loadMs": 0,
70
+ "inferenceMs": 59.90000009536743,
71
+ "disposeMs": 0,
72
+ "totalMs": 60.10000038146973
73
+ },
74
+ {
75
+ "loadMs": 0,
76
+ "inferenceMs": 60,
77
+ "disposeMs": 0,
78
+ "totalMs": 60.09999990463257
79
+ },
80
+ {
81
+ "loadMs": 0,
82
+ "inferenceMs": 61.700000286102295,
83
+ "disposeMs": 0,
84
+ "totalMs": 61.90000009536743
85
+ },
86
+ {
87
+ "loadMs": 0,
88
+ "inferenceMs": 64.30000019073486,
89
+ "disposeMs": 0,
90
+ "totalMs": 64.40000009536743
91
+ },
92
+ {
93
+ "loadMs": 0,
94
+ "inferenceMs": 62.09999990463257,
95
+ "disposeMs": 0,
96
+ "totalMs": 62.19999980926514
97
+ },
98
+ {
99
+ "loadMs": 0,
100
+ "inferenceMs": 61.5,
101
+ "disposeMs": 0,
102
+ "totalMs": 61.5
103
+ },
104
+ {
105
+ "loadMs": 0,
106
+ "inferenceMs": 60.59999990463257,
107
+ "disposeMs": 0,
108
+ "totalMs": 60.69999980926514
109
+ },
110
+ {
111
+ "loadMs": 0,
112
+ "inferenceMs": 60.799999713897705,
113
+ "disposeMs": 0,
114
+ "totalMs": 60.89999961853027
115
+ },
116
+ {
117
+ "loadMs": 0,
118
+ "inferenceMs": 60.5,
119
+ "disposeMs": 0,
120
+ "totalMs": 60.59999990463257
121
+ },
122
+ {
123
+ "loadMs": 0,
124
+ "inferenceMs": 61.59999990463257,
125
+ "disposeMs": 0,
126
+ "totalMs": 61.59999990463257
127
+ }
128
+ ],
129
+ "first": {
130
+ "loadMs": 357.5,
131
+ "inferenceMs": 67.80000019073486,
132
+ "disposeMs": 0,
133
+ "totalMs": 425.7000002861023
134
+ }
135
+ },
136
+ {
137
+ "size": "17m",
138
+ "device": "cpu",
139
+ "dtype": "fp32",
140
+ "samples": [
141
+ {
142
+ "loadMs": 0,
143
+ "inferenceMs": 57,
144
+ "disposeMs": 0,
145
+ "totalMs": 57.09999990463257
146
+ },
147
+ {
148
+ "loadMs": 0,
149
+ "inferenceMs": 55.40000009536743,
150
+ "disposeMs": 0,
151
+ "totalMs": 55.59999990463257
152
+ },
153
+ {
154
+ "loadMs": 0,
155
+ "inferenceMs": 56.799999713897705,
156
+ "disposeMs": 0,
157
+ "totalMs": 56.799999713897705
158
+ },
159
+ {
160
+ "loadMs": 0,
161
+ "inferenceMs": 60.19999980926514,
162
+ "disposeMs": 0,
163
+ "totalMs": 60.5
164
+ },
165
+ {
166
+ "loadMs": 0,
167
+ "inferenceMs": 58.5,
168
+ "disposeMs": 0,
169
+ "totalMs": 58.60000038146973
170
+ },
171
+ {
172
+ "loadMs": 0,
173
+ "inferenceMs": 59.700000286102295,
174
+ "disposeMs": 0,
175
+ "totalMs": 59.80000019073486
176
+ },
177
+ {
178
+ "loadMs": 0,
179
+ "inferenceMs": 69.5,
180
+ "disposeMs": 0,
181
+ "totalMs": 69.60000038146973
182
+ },
183
+ {
184
+ "loadMs": 0,
185
+ "inferenceMs": 82,
186
+ "disposeMs": 0,
187
+ "totalMs": 82.2000002861023
188
+ },
189
+ {
190
+ "loadMs": 0,
191
+ "inferenceMs": 59.40000009536743,
192
+ "disposeMs": 0,
193
+ "totalMs": 59.59999990463257
194
+ },
195
+ {
196
+ "loadMs": 0,
197
+ "inferenceMs": 58.40000009536743,
198
+ "disposeMs": 0,
199
+ "totalMs": 58.5
200
+ },
201
+ {
202
+ "loadMs": 0,
203
+ "inferenceMs": 57.200000286102295,
204
+ "disposeMs": 0,
205
+ "totalMs": 57.200000286102295
206
+ },
207
+ {
208
+ "loadMs": 0,
209
+ "inferenceMs": 56.19999980926514,
210
+ "disposeMs": 0,
211
+ "totalMs": 56.19999980926514
212
+ },
213
+ {
214
+ "loadMs": 0,
215
+ "inferenceMs": 58,
216
+ "disposeMs": 0,
217
+ "totalMs": 58.09999990463257
218
+ },
219
+ {
220
+ "loadMs": 0,
221
+ "inferenceMs": 57.59999990463257,
222
+ "disposeMs": 0,
223
+ "totalMs": 57.69999980926514
224
+ },
225
+ {
226
+ "loadMs": 0,
227
+ "inferenceMs": 56.200000286102295,
228
+ "disposeMs": 0,
229
+ "totalMs": 56.30000019073486
230
+ },
231
+ {
232
+ "loadMs": 0,
233
+ "inferenceMs": 56.30000019073486,
234
+ "disposeMs": 0,
235
+ "totalMs": 56.40000009536743
236
+ },
237
+ {
238
+ "loadMs": 0,
239
+ "inferenceMs": 55.299999713897705,
240
+ "disposeMs": 0,
241
+ "totalMs": 55.299999713897705
242
+ },
243
+ {
244
+ "loadMs": 0,
245
+ "inferenceMs": 56.799999713897705,
246
+ "disposeMs": 0,
247
+ "totalMs": 57
248
+ },
249
+ {
250
+ "loadMs": 0,
251
+ "inferenceMs": 56.30000019073486,
252
+ "disposeMs": 0,
253
+ "totalMs": 56.30000019073486
254
+ },
255
+ {
256
+ "loadMs": 0,
257
+ "inferenceMs": 56.19999980926514,
258
+ "disposeMs": 0,
259
+ "totalMs": 56.299999713897705
260
+ }
261
+ ],
262
+ "first": {
263
+ "loadMs": 189.90000009536743,
264
+ "inferenceMs": 60.40000009536743,
265
+ "disposeMs": 0,
266
+ "totalMs": 253.30000019073486
267
+ }
268
+ },
269
+ {
270
+ "size": "17m",
271
+ "device": "gpu",
272
+ "dtype": "fp32",
273
+ "samples": [
274
+ {
275
+ "loadMs": 0,
276
+ "inferenceMs": 13.5,
277
+ "disposeMs": 0,
278
+ "totalMs": 13.599999904632568
279
+ },
280
+ {
281
+ "loadMs": 0,
282
+ "inferenceMs": 9.299999713897705,
283
+ "disposeMs": 0,
284
+ "totalMs": 9.299999713897705
285
+ },
286
+ {
287
+ "loadMs": 0,
288
+ "inferenceMs": 12.300000190734863,
289
+ "disposeMs": 0,
290
+ "totalMs": 12.400000095367432
291
+ },
292
+ {
293
+ "loadMs": 0,
294
+ "inferenceMs": 15.900000095367432,
295
+ "disposeMs": 0,
296
+ "totalMs": 15.900000095367432
297
+ },
298
+ {
299
+ "loadMs": 0,
300
+ "inferenceMs": 9.300000190734863,
301
+ "disposeMs": 0,
302
+ "totalMs": 9.5
303
+ },
304
+ {
305
+ "loadMs": 0,
306
+ "inferenceMs": 15.599999904632568,
307
+ "disposeMs": 0,
308
+ "totalMs": 15.700000286102295
309
+ },
310
+ {
311
+ "loadMs": 0,
312
+ "inferenceMs": 14,
313
+ "disposeMs": 0,
314
+ "totalMs": 14.200000286102295
315
+ },
316
+ {
317
+ "loadMs": 0,
318
+ "inferenceMs": 12.800000190734863,
319
+ "disposeMs": 0,
320
+ "totalMs": 12.800000190734863
321
+ },
322
+ {
323
+ "loadMs": 0,
324
+ "inferenceMs": 10.900000095367432,
325
+ "disposeMs": 0,
326
+ "totalMs": 10.900000095367432
327
+ },
328
+ {
329
+ "loadMs": 0,
330
+ "inferenceMs": 10,
331
+ "disposeMs": 0,
332
+ "totalMs": 10.099999904632568
333
+ },
334
+ {
335
+ "loadMs": 0,
336
+ "inferenceMs": 16.09999990463257,
337
+ "disposeMs": 0,
338
+ "totalMs": 16.09999990463257
339
+ },
340
+ {
341
+ "loadMs": 0,
342
+ "inferenceMs": 14.800000190734863,
343
+ "disposeMs": 0,
344
+ "totalMs": 14.900000095367432
345
+ },
346
+ {
347
+ "loadMs": 0,
348
+ "inferenceMs": 11,
349
+ "disposeMs": 0,
350
+ "totalMs": 11.200000286102295
351
+ },
352
+ {
353
+ "loadMs": 0,
354
+ "inferenceMs": 9.199999809265137,
355
+ "disposeMs": 0,
356
+ "totalMs": 9.299999713897705
357
+ },
358
+ {
359
+ "loadMs": 0,
360
+ "inferenceMs": 11.599999904632568,
361
+ "disposeMs": 0,
362
+ "totalMs": 11.599999904632568
363
+ },
364
+ {
365
+ "loadMs": 0,
366
+ "inferenceMs": 13.700000286102295,
367
+ "disposeMs": 0,
368
+ "totalMs": 13.700000286102295
369
+ },
370
+ {
371
+ "loadMs": 0,
372
+ "inferenceMs": 9.599999904632568,
373
+ "disposeMs": 0,
374
+ "totalMs": 9.699999809265137
375
+ },
376
+ {
377
+ "loadMs": 0,
378
+ "inferenceMs": 9.5,
379
+ "disposeMs": 0,
380
+ "totalMs": 9.5
381
+ },
382
+ {
383
+ "loadMs": 0,
384
+ "inferenceMs": 16.40000009536743,
385
+ "disposeMs": 0,
386
+ "totalMs": 16.5
387
+ },
388
+ {
389
+ "loadMs": 0,
390
+ "inferenceMs": 15.699999809265137,
391
+ "disposeMs": 0,
392
+ "totalMs": 15.699999809265137
393
+ }
394
+ ],
395
+ "first": {
396
+ "loadMs": 215,
397
+ "inferenceMs": 66,
398
+ "disposeMs": 0,
399
+ "totalMs": 282.30000019073486
400
+ }
401
+ },
402
+ {
403
+ "size": "32m",
404
+ "device": "cpu",
405
+ "dtype": "q8",
406
+ "samples": [
407
+ {
408
+ "loadMs": 0,
409
+ "inferenceMs": 193.09999990463257,
410
+ "disposeMs": 0,
411
+ "totalMs": 193.2000002861023
412
+ },
413
+ {
414
+ "loadMs": 0,
415
+ "inferenceMs": 187,
416
+ "disposeMs": 0,
417
+ "totalMs": 187
418
+ },
419
+ {
420
+ "loadMs": 0,
421
+ "inferenceMs": 233,
422
+ "disposeMs": 0,
423
+ "totalMs": 233.09999990463257
424
+ },
425
+ {
426
+ "loadMs": 0,
427
+ "inferenceMs": 183.90000009536743,
428
+ "disposeMs": 0,
429
+ "totalMs": 184.09999990463257
430
+ },
431
+ {
432
+ "loadMs": 0,
433
+ "inferenceMs": 181.5,
434
+ "disposeMs": 0,
435
+ "totalMs": 181.60000038146973
436
+ },
437
+ {
438
+ "loadMs": 0,
439
+ "inferenceMs": 215.19999980926514,
440
+ "disposeMs": 0,
441
+ "totalMs": 215.19999980926514
442
+ },
443
+ {
444
+ "loadMs": 0,
445
+ "inferenceMs": 183,
446
+ "disposeMs": 0,
447
+ "totalMs": 183
448
+ },
449
+ {
450
+ "loadMs": 0,
451
+ "inferenceMs": 186.60000038146973,
452
+ "disposeMs": 0,
453
+ "totalMs": 186.60000038146973
454
+ },
455
+ {
456
+ "loadMs": 0,
457
+ "inferenceMs": 187.40000009536743,
458
+ "disposeMs": 0,
459
+ "totalMs": 187.5
460
+ },
461
+ {
462
+ "loadMs": 0,
463
+ "inferenceMs": 184.7999997138977,
464
+ "disposeMs": 0,
465
+ "totalMs": 184.7999997138977
466
+ },
467
+ {
468
+ "loadMs": 0,
469
+ "inferenceMs": 187.5,
470
+ "disposeMs": 0,
471
+ "totalMs": 187.5
472
+ },
473
+ {
474
+ "loadMs": 0,
475
+ "inferenceMs": 187.90000009536743,
476
+ "disposeMs": 0,
477
+ "totalMs": 188
478
+ },
479
+ {
480
+ "loadMs": 0,
481
+ "inferenceMs": 189.40000009536743,
482
+ "disposeMs": 0,
483
+ "totalMs": 189.40000009536743
484
+ },
485
+ {
486
+ "loadMs": 0,
487
+ "inferenceMs": 192.90000009536743,
488
+ "disposeMs": 0,
489
+ "totalMs": 192.90000009536743
490
+ },
491
+ {
492
+ "loadMs": 0,
493
+ "inferenceMs": 186.7999997138977,
494
+ "disposeMs": 0,
495
+ "totalMs": 186.7999997138977
496
+ },
497
+ {
498
+ "loadMs": 0,
499
+ "inferenceMs": 187.09999990463257,
500
+ "disposeMs": 0,
501
+ "totalMs": 187.09999990463257
502
+ },
503
+ {
504
+ "loadMs": 0,
505
+ "inferenceMs": 185.7999997138977,
506
+ "disposeMs": 0,
507
+ "totalMs": 185.90000009536743
508
+ },
509
+ {
510
+ "loadMs": 0,
511
+ "inferenceMs": 188,
512
+ "disposeMs": 0,
513
+ "totalMs": 188
514
+ },
515
+ {
516
+ "loadMs": 0,
517
+ "inferenceMs": 183.7000002861023,
518
+ "disposeMs": 0,
519
+ "totalMs": 183.7000002861023
520
+ },
521
+ {
522
+ "loadMs": 0,
523
+ "inferenceMs": 185.09999990463257,
524
+ "disposeMs": 0,
525
+ "totalMs": 185.19999980926514
526
+ }
527
+ ],
528
+ "first": {
529
+ "loadMs": 221.69999980926514,
530
+ "inferenceMs": 189.7000002861023,
531
+ "disposeMs": 0,
532
+ "totalMs": 413.40000009536743
533
+ }
534
+ },
535
+ {
536
+ "size": "32m",
537
+ "device": "cpu",
538
+ "dtype": "fp32",
539
+ "samples": [
540
+ {
541
+ "loadMs": 0,
542
+ "inferenceMs": 167.60000038146973,
543
+ "disposeMs": 0,
544
+ "totalMs": 167.60000038146973
545
+ },
546
+ {
547
+ "loadMs": 0,
548
+ "inferenceMs": 166.5,
549
+ "disposeMs": 0,
550
+ "totalMs": 166.5
551
+ },
552
+ {
553
+ "loadMs": 0,
554
+ "inferenceMs": 165.60000038146973,
555
+ "disposeMs": 0,
556
+ "totalMs": 165.7000002861023
557
+ },
558
+ {
559
+ "loadMs": 0,
560
+ "inferenceMs": 168.2999997138977,
561
+ "disposeMs": 0,
562
+ "totalMs": 168.39999961853027
563
+ },
564
+ {
565
+ "loadMs": 0,
566
+ "inferenceMs": 171.59999990463257,
567
+ "disposeMs": 0,
568
+ "totalMs": 171.7000002861023
569
+ },
570
+ {
571
+ "loadMs": 0,
572
+ "inferenceMs": 166.2999997138977,
573
+ "disposeMs": 0,
574
+ "totalMs": 166.5
575
+ },
576
+ {
577
+ "loadMs": 0,
578
+ "inferenceMs": 165.09999990463257,
579
+ "disposeMs": 0,
580
+ "totalMs": 165.09999990463257
581
+ },
582
+ {
583
+ "loadMs": 0,
584
+ "inferenceMs": 200,
585
+ "disposeMs": 0,
586
+ "totalMs": 200
587
+ },
588
+ {
589
+ "loadMs": 0,
590
+ "inferenceMs": 164.5,
591
+ "disposeMs": 0,
592
+ "totalMs": 164.60000038146973
593
+ },
594
+ {
595
+ "loadMs": 0,
596
+ "inferenceMs": 166.5,
597
+ "disposeMs": 0,
598
+ "totalMs": 166.69999980926514
599
+ },
600
+ {
601
+ "loadMs": 0,
602
+ "inferenceMs": 168.59999990463257,
603
+ "disposeMs": 0,
604
+ "totalMs": 168.7000002861023
605
+ },
606
+ {
607
+ "loadMs": 0,
608
+ "inferenceMs": 170.09999990463257,
609
+ "disposeMs": 0,
610
+ "totalMs": 170.30000019073486
611
+ },
612
+ {
613
+ "loadMs": 0,
614
+ "inferenceMs": 172,
615
+ "disposeMs": 0,
616
+ "totalMs": 172.19999980926514
617
+ },
618
+ {
619
+ "loadMs": 0,
620
+ "inferenceMs": 170.80000019073486,
621
+ "disposeMs": 0,
622
+ "totalMs": 170.80000019073486
623
+ },
624
+ {
625
+ "loadMs": 0,
626
+ "inferenceMs": 168.39999961853027,
627
+ "disposeMs": 0,
628
+ "totalMs": 168.5
629
+ },
630
+ {
631
+ "loadMs": 0,
632
+ "inferenceMs": 169.89999961853027,
633
+ "disposeMs": 0,
634
+ "totalMs": 170
635
+ },
636
+ {
637
+ "loadMs": 0,
638
+ "inferenceMs": 170.5,
639
+ "disposeMs": 0,
640
+ "totalMs": 170.5
641
+ },
642
+ {
643
+ "loadMs": 0,
644
+ "inferenceMs": 163.80000019073486,
645
+ "disposeMs": 0,
646
+ "totalMs": 163.90000009536743
647
+ },
648
+ {
649
+ "loadMs": 0,
650
+ "inferenceMs": 164.2999997138977,
651
+ "disposeMs": 0,
652
+ "totalMs": 164.2999997138977
653
+ },
654
+ {
655
+ "loadMs": 0,
656
+ "inferenceMs": 166.60000038146973,
657
+ "disposeMs": 0,
658
+ "totalMs": 166.80000019073486
659
+ }
660
+ ],
661
+ "first": {
662
+ "loadMs": 262.40000009536743,
663
+ "inferenceMs": 173.7999997138977,
664
+ "disposeMs": 0,
665
+ "totalMs": 438.69999980926514
666
+ }
667
+ },
668
+ {
669
+ "size": "32m",
670
+ "device": "gpu",
671
+ "dtype": "fp32",
672
+ "samples": [
673
+ {
674
+ "loadMs": 0,
675
+ "inferenceMs": 16.700000286102295,
676
+ "disposeMs": 0,
677
+ "totalMs": 16.700000286102295
678
+ },
679
+ {
680
+ "loadMs": 0,
681
+ "inferenceMs": 16.399999618530273,
682
+ "disposeMs": 0,
683
+ "totalMs": 16.399999618530273
684
+ },
685
+ {
686
+ "loadMs": 0,
687
+ "inferenceMs": 16.40000009536743,
688
+ "disposeMs": 0,
689
+ "totalMs": 16.40000009536743
690
+ },
691
+ {
692
+ "loadMs": 0,
693
+ "inferenceMs": 14.199999809265137,
694
+ "disposeMs": 0,
695
+ "totalMs": 14.400000095367432
696
+ },
697
+ {
698
+ "loadMs": 0,
699
+ "inferenceMs": 13.799999713897705,
700
+ "disposeMs": 0,
701
+ "totalMs": 13.799999713897705
702
+ },
703
+ {
704
+ "loadMs": 0,
705
+ "inferenceMs": 14.799999713897705,
706
+ "disposeMs": 0,
707
+ "totalMs": 14.900000095367432
708
+ },
709
+ {
710
+ "loadMs": 0,
711
+ "inferenceMs": 16.199999809265137,
712
+ "disposeMs": 0,
713
+ "totalMs": 16.199999809265137
714
+ },
715
+ {
716
+ "loadMs": 0,
717
+ "inferenceMs": 15.299999713897705,
718
+ "disposeMs": 0,
719
+ "totalMs": 15.399999618530273
720
+ },
721
+ {
722
+ "loadMs": 0,
723
+ "inferenceMs": 14.699999809265137,
724
+ "disposeMs": 0,
725
+ "totalMs": 14.699999809265137
726
+ },
727
+ {
728
+ "loadMs": 0,
729
+ "inferenceMs": 13,
730
+ "disposeMs": 0,
731
+ "totalMs": 13.099999904632568
732
+ },
733
+ {
734
+ "loadMs": 0,
735
+ "inferenceMs": 12.300000190734863,
736
+ "disposeMs": 0,
737
+ "totalMs": 12.5
738
+ },
739
+ {
740
+ "loadMs": 0,
741
+ "inferenceMs": 12.200000286102295,
742
+ "disposeMs": 0,
743
+ "totalMs": 12.300000190734863
744
+ },
745
+ {
746
+ "loadMs": 0,
747
+ "inferenceMs": 12.199999809265137,
748
+ "disposeMs": 0,
749
+ "totalMs": 12.199999809265137
750
+ },
751
+ {
752
+ "loadMs": 0,
753
+ "inferenceMs": 12.5,
754
+ "disposeMs": 0,
755
+ "totalMs": 12.599999904632568
756
+ },
757
+ {
758
+ "loadMs": 0,
759
+ "inferenceMs": 14.800000190734863,
760
+ "disposeMs": 0,
761
+ "totalMs": 14.800000190734863
762
+ },
763
+ {
764
+ "loadMs": 0,
765
+ "inferenceMs": 14.700000286102295,
766
+ "disposeMs": 0,
767
+ "totalMs": 14.800000190734863
768
+ },
769
+ {
770
+ "loadMs": 0,
771
+ "inferenceMs": 14.599999904632568,
772
+ "disposeMs": 0,
773
+ "totalMs": 14.699999809265137
774
+ },
775
+ {
776
+ "loadMs": 0,
777
+ "inferenceMs": 15.099999904632568,
778
+ "disposeMs": 0,
779
+ "totalMs": 15.099999904632568
780
+ },
781
+ {
782
+ "loadMs": 0,
783
+ "inferenceMs": 13.099999904632568,
784
+ "disposeMs": 0,
785
+ "totalMs": 13.199999809265137
786
+ },
787
+ {
788
+ "loadMs": 0,
789
+ "inferenceMs": 13,
790
+ "disposeMs": 0,
791
+ "totalMs": 13
792
+ }
793
+ ],
794
+ "first": {
795
+ "loadMs": 301.7999997138977,
796
+ "inferenceMs": 24.90000009536743,
797
+ "disposeMs": 0,
798
+ "totalMs": 328.7999997138977
799
+ }
800
+ },
801
+ {
802
+ "size": "68m",
803
+ "device": "cpu",
804
+ "dtype": "q8",
805
+ "samples": [
806
+ {
807
+ "loadMs": 0,
808
+ "inferenceMs": 594.3000001907349,
809
+ "disposeMs": 0,
810
+ "totalMs": 594.4000000953674
811
+ },
812
+ {
813
+ "loadMs": 0,
814
+ "inferenceMs": 595.3000001907349,
815
+ "disposeMs": 0,
816
+ "totalMs": 595.4000000953674
817
+ },
818
+ {
819
+ "loadMs": 0,
820
+ "inferenceMs": 573.3000001907349,
821
+ "disposeMs": 0,
822
+ "totalMs": 573.5
823
+ },
824
+ {
825
+ "loadMs": 0,
826
+ "inferenceMs": 575.9000000953674,
827
+ "disposeMs": 0,
828
+ "totalMs": 575.9000000953674
829
+ },
830
+ {
831
+ "loadMs": 0,
832
+ "inferenceMs": 621.5999999046326,
833
+ "disposeMs": 0,
834
+ "totalMs": 621.6999998092651
835
+ },
836
+ {
837
+ "loadMs": 0,
838
+ "inferenceMs": 575,
839
+ "disposeMs": 0,
840
+ "totalMs": 575
841
+ },
842
+ {
843
+ "loadMs": 0,
844
+ "inferenceMs": 582.3000001907349,
845
+ "disposeMs": 0,
846
+ "totalMs": 582.3000001907349
847
+ },
848
+ {
849
+ "loadMs": 0,
850
+ "inferenceMs": 571.9000000953674,
851
+ "disposeMs": 0,
852
+ "totalMs": 571.9000000953674
853
+ },
854
+ {
855
+ "loadMs": 0,
856
+ "inferenceMs": 574.8000001907349,
857
+ "disposeMs": 0,
858
+ "totalMs": 574.9000000953674
859
+ },
860
+ {
861
+ "loadMs": 0,
862
+ "inferenceMs": 576.8000001907349,
863
+ "disposeMs": 0,
864
+ "totalMs": 576.9000000953674
865
+ },
866
+ {
867
+ "loadMs": 0,
868
+ "inferenceMs": 578.4000000953674,
869
+ "disposeMs": 0,
870
+ "totalMs": 578.5999999046326
871
+ },
872
+ {
873
+ "loadMs": 0,
874
+ "inferenceMs": 590.8000001907349,
875
+ "disposeMs": 0,
876
+ "totalMs": 590.8000001907349
877
+ },
878
+ {
879
+ "loadMs": 0,
880
+ "inferenceMs": 574.5999999046326,
881
+ "disposeMs": 0,
882
+ "totalMs": 574.7999997138977
883
+ },
884
+ {
885
+ "loadMs": 0,
886
+ "inferenceMs": 596.7999997138977,
887
+ "disposeMs": 0,
888
+ "totalMs": 596.9000000953674
889
+ },
890
+ {
891
+ "loadMs": 0,
892
+ "inferenceMs": 573.8999996185303,
893
+ "disposeMs": 0,
894
+ "totalMs": 574
895
+ },
896
+ {
897
+ "loadMs": 0,
898
+ "inferenceMs": 571.6999998092651,
899
+ "disposeMs": 0,
900
+ "totalMs": 571.6999998092651
901
+ },
902
+ {
903
+ "loadMs": 0,
904
+ "inferenceMs": 572.0999999046326,
905
+ "disposeMs": 0,
906
+ "totalMs": 572.3000001907349
907
+ },
908
+ {
909
+ "loadMs": 0,
910
+ "inferenceMs": 582.9000000953674,
911
+ "disposeMs": 0,
912
+ "totalMs": 583
913
+ },
914
+ {
915
+ "loadMs": 0,
916
+ "inferenceMs": 579.6999998092651,
917
+ "disposeMs": 0,
918
+ "totalMs": 579.8999996185303
919
+ },
920
+ {
921
+ "loadMs": 0,
922
+ "inferenceMs": 584,
923
+ "disposeMs": 0,
924
+ "totalMs": 584
925
+ }
926
+ ],
927
+ "first": {
928
+ "loadMs": 424,
929
+ "inferenceMs": 601.4000000953674,
930
+ "disposeMs": 0,
931
+ "totalMs": 1027.3999996185303
932
+ }
933
+ },
934
+ {
935
+ "size": "68m",
936
+ "device": "cpu",
937
+ "dtype": "fp32",
938
+ "samples": [
939
+ {
940
+ "loadMs": 0,
941
+ "inferenceMs": 551.7999997138977,
942
+ "disposeMs": 0,
943
+ "totalMs": 551.8999996185303
944
+ },
945
+ {
946
+ "loadMs": 0,
947
+ "inferenceMs": 523.8000001907349,
948
+ "disposeMs": 0,
949
+ "totalMs": 523.8000001907349
950
+ },
951
+ {
952
+ "loadMs": 0,
953
+ "inferenceMs": 534.9000000953674,
954
+ "disposeMs": 0,
955
+ "totalMs": 534.9000000953674
956
+ },
957
+ {
958
+ "loadMs": 0,
959
+ "inferenceMs": 541.6999998092651,
960
+ "disposeMs": 0,
961
+ "totalMs": 541.9000000953674
962
+ },
963
+ {
964
+ "loadMs": 0,
965
+ "inferenceMs": 529.2999997138977,
966
+ "disposeMs": 0,
967
+ "totalMs": 529.2999997138977
968
+ },
969
+ {
970
+ "loadMs": 0,
971
+ "inferenceMs": 526.0999999046326,
972
+ "disposeMs": 0,
973
+ "totalMs": 526.0999999046326
974
+ },
975
+ {
976
+ "loadMs": 0,
977
+ "inferenceMs": 550.2999997138977,
978
+ "disposeMs": 0,
979
+ "totalMs": 550.3999996185303
980
+ },
981
+ {
982
+ "loadMs": 0,
983
+ "inferenceMs": 532.9000000953674,
984
+ "disposeMs": 0,
985
+ "totalMs": 533
986
+ },
987
+ {
988
+ "loadMs": 0,
989
+ "inferenceMs": 524.5,
990
+ "disposeMs": 0,
991
+ "totalMs": 524.6999998092651
992
+ },
993
+ {
994
+ "loadMs": 0,
995
+ "inferenceMs": 541.9000000953674,
996
+ "disposeMs": 0,
997
+ "totalMs": 541.9000000953674
998
+ },
999
+ {
1000
+ "loadMs": 0,
1001
+ "inferenceMs": 542.5,
1002
+ "disposeMs": 0,
1003
+ "totalMs": 542.5999999046326
1004
+ },
1005
+ {
1006
+ "loadMs": 0,
1007
+ "inferenceMs": 536.5999999046326,
1008
+ "disposeMs": 0,
1009
+ "totalMs": 536.7999997138977
1010
+ },
1011
+ {
1012
+ "loadMs": 0,
1013
+ "inferenceMs": 545.6000003814697,
1014
+ "disposeMs": 0,
1015
+ "totalMs": 545.6000003814697
1016
+ },
1017
+ {
1018
+ "loadMs": 0,
1019
+ "inferenceMs": 535,
1020
+ "disposeMs": 0,
1021
+ "totalMs": 535
1022
+ },
1023
+ {
1024
+ "loadMs": 0,
1025
+ "inferenceMs": 530,
1026
+ "disposeMs": 0,
1027
+ "totalMs": 530
1028
+ },
1029
+ {
1030
+ "loadMs": 0,
1031
+ "inferenceMs": 585.5999999046326,
1032
+ "disposeMs": 0,
1033
+ "totalMs": 585.7000002861023
1034
+ },
1035
+ {
1036
+ "loadMs": 0,
1037
+ "inferenceMs": 523.2999997138977,
1038
+ "disposeMs": 0,
1039
+ "totalMs": 523.2999997138977
1040
+ },
1041
+ {
1042
+ "loadMs": 0,
1043
+ "inferenceMs": 527.5,
1044
+ "disposeMs": 0,
1045
+ "totalMs": 527.6000003814697
1046
+ },
1047
+ {
1048
+ "loadMs": 0,
1049
+ "inferenceMs": 525.1999998092651,
1050
+ "disposeMs": 0,
1051
+ "totalMs": 525.1999998092651
1052
+ },
1053
+ {
1054
+ "loadMs": 0,
1055
+ "inferenceMs": 536,
1056
+ "disposeMs": 0,
1057
+ "totalMs": 536
1058
+ }
1059
+ ],
1060
+ "first": {
1061
+ "loadMs": 673.7000002861023,
1062
+ "inferenceMs": 549.5999999046326,
1063
+ "disposeMs": 0,
1064
+ "totalMs": 1227
1065
+ }
1066
+ },
1067
+ {
1068
+ "size": "68m",
1069
+ "device": "gpu",
1070
+ "dtype": "fp32",
1071
+ "samples": [
1072
+ {
1073
+ "loadMs": 0,
1074
+ "inferenceMs": 31.600000381469727,
1075
+ "disposeMs": 0,
1076
+ "totalMs": 31.600000381469727
1077
+ },
1078
+ {
1079
+ "loadMs": 0,
1080
+ "inferenceMs": 31.800000190734863,
1081
+ "disposeMs": 0,
1082
+ "totalMs": 31.90000009536743
1083
+ },
1084
+ {
1085
+ "loadMs": 0,
1086
+ "inferenceMs": 32.09999990463257,
1087
+ "disposeMs": 0,
1088
+ "totalMs": 32.299999713897705
1089
+ },
1090
+ {
1091
+ "loadMs": 0,
1092
+ "inferenceMs": 31.59999990463257,
1093
+ "disposeMs": 0,
1094
+ "totalMs": 31.699999809265137
1095
+ },
1096
+ {
1097
+ "loadMs": 0,
1098
+ "inferenceMs": 31.200000286102295,
1099
+ "disposeMs": 0,
1100
+ "totalMs": 31.200000286102295
1101
+ },
1102
+ {
1103
+ "loadMs": 0,
1104
+ "inferenceMs": 32.5,
1105
+ "disposeMs": 0,
1106
+ "totalMs": 32.60000038146973
1107
+ },
1108
+ {
1109
+ "loadMs": 0,
1110
+ "inferenceMs": 31.800000190734863,
1111
+ "disposeMs": 0,
1112
+ "totalMs": 31.800000190734863
1113
+ },
1114
+ {
1115
+ "loadMs": 0,
1116
+ "inferenceMs": 32.59999990463257,
1117
+ "disposeMs": 0,
1118
+ "totalMs": 32.69999980926514
1119
+ },
1120
+ {
1121
+ "loadMs": 0,
1122
+ "inferenceMs": 37,
1123
+ "disposeMs": 0,
1124
+ "totalMs": 37.10000038146973
1125
+ },
1126
+ {
1127
+ "loadMs": 0,
1128
+ "inferenceMs": 36.30000019073486,
1129
+ "disposeMs": 0,
1130
+ "totalMs": 36.30000019073486
1131
+ },
1132
+ {
1133
+ "loadMs": 0,
1134
+ "inferenceMs": 33.299999713897705,
1135
+ "disposeMs": 0,
1136
+ "totalMs": 33.299999713897705
1137
+ },
1138
+ {
1139
+ "loadMs": 0,
1140
+ "inferenceMs": 33.700000286102295,
1141
+ "disposeMs": 0,
1142
+ "totalMs": 33.80000019073486
1143
+ },
1144
+ {
1145
+ "loadMs": 0,
1146
+ "inferenceMs": 33.299999713897705,
1147
+ "disposeMs": 0,
1148
+ "totalMs": 33.5
1149
+ },
1150
+ {
1151
+ "loadMs": 0,
1152
+ "inferenceMs": 34,
1153
+ "disposeMs": 0,
1154
+ "totalMs": 34
1155
+ },
1156
+ {
1157
+ "loadMs": 0,
1158
+ "inferenceMs": 35.90000009536743,
1159
+ "disposeMs": 0,
1160
+ "totalMs": 35.90000009536743
1161
+ },
1162
+ {
1163
+ "loadMs": 0,
1164
+ "inferenceMs": 32.80000019073486,
1165
+ "disposeMs": 0,
1166
+ "totalMs": 32.80000019073486
1167
+ },
1168
+ {
1169
+ "loadMs": 0,
1170
+ "inferenceMs": 33.09999990463257,
1171
+ "disposeMs": 0,
1172
+ "totalMs": 33.19999980926514
1173
+ },
1174
+ {
1175
+ "loadMs": 0,
1176
+ "inferenceMs": 33.30000019073486,
1177
+ "disposeMs": 0,
1178
+ "totalMs": 33.40000009536743
1179
+ },
1180
+ {
1181
+ "loadMs": 0,
1182
+ "inferenceMs": 33.5,
1183
+ "disposeMs": 0,
1184
+ "totalMs": 33.59999990463257
1185
+ },
1186
+ {
1187
+ "loadMs": 0,
1188
+ "inferenceMs": 35.5,
1189
+ "disposeMs": 0,
1190
+ "totalMs": 35.5
1191
+ }
1192
+ ],
1193
+ "first": {
1194
+ "loadMs": 748.5,
1195
+ "inferenceMs": 47,
1196
+ "disposeMs": 0,
1197
+ "totalMs": 798.8000001907349
1198
+ }
1199
+ },
1200
+ {
1201
+ "size": "150m",
1202
+ "device": "cpu",
1203
+ "dtype": "q8",
1204
+ "samples": [
1205
+ {
1206
+ "loadMs": 0,
1207
+ "inferenceMs": 1476.7999997138977,
1208
+ "disposeMs": 0,
1209
+ "totalMs": 1476.7999997138977
1210
+ },
1211
+ {
1212
+ "loadMs": 0,
1213
+ "inferenceMs": 1464.0999999046326,
1214
+ "disposeMs": 0,
1215
+ "totalMs": 1464.2000002861023
1216
+ },
1217
+ {
1218
+ "loadMs": 0,
1219
+ "inferenceMs": 1485.5,
1220
+ "disposeMs": 0,
1221
+ "totalMs": 1485.5999999046326
1222
+ },
1223
+ {
1224
+ "loadMs": 0,
1225
+ "inferenceMs": 1465.8000001907349,
1226
+ "disposeMs": 0,
1227
+ "totalMs": 1465.8000001907349
1228
+ },
1229
+ {
1230
+ "loadMs": 0,
1231
+ "inferenceMs": 1492.5999999046326,
1232
+ "disposeMs": 0,
1233
+ "totalMs": 1492.5999999046326
1234
+ },
1235
+ {
1236
+ "loadMs": 0,
1237
+ "inferenceMs": 1475,
1238
+ "disposeMs": 0,
1239
+ "totalMs": 1475.0999999046326
1240
+ },
1241
+ {
1242
+ "loadMs": 0,
1243
+ "inferenceMs": 1461.4000000953674,
1244
+ "disposeMs": 0,
1245
+ "totalMs": 1461.5999999046326
1246
+ },
1247
+ {
1248
+ "loadMs": 0,
1249
+ "inferenceMs": 1466.5999999046326,
1250
+ "disposeMs": 0,
1251
+ "totalMs": 1466.7000002861023
1252
+ },
1253
+ {
1254
+ "loadMs": 0,
1255
+ "inferenceMs": 1626.5999999046326,
1256
+ "disposeMs": 0,
1257
+ "totalMs": 1626.6999998092651
1258
+ },
1259
+ {
1260
+ "loadMs": 0,
1261
+ "inferenceMs": 1481.6999998092651,
1262
+ "disposeMs": 0,
1263
+ "totalMs": 1481.6999998092651
1264
+ },
1265
+ {
1266
+ "loadMs": 0,
1267
+ "inferenceMs": 1470.0999999046326,
1268
+ "disposeMs": 0,
1269
+ "totalMs": 1470.1999998092651
1270
+ },
1271
+ {
1272
+ "loadMs": 0,
1273
+ "inferenceMs": 1457,
1274
+ "disposeMs": 0,
1275
+ "totalMs": 1457.1999998092651
1276
+ },
1277
+ {
1278
+ "loadMs": 0,
1279
+ "inferenceMs": 1526.9000000953674,
1280
+ "disposeMs": 0,
1281
+ "totalMs": 1526.9000000953674
1282
+ },
1283
+ {
1284
+ "loadMs": 0,
1285
+ "inferenceMs": 1454.5999999046326,
1286
+ "disposeMs": 0,
1287
+ "totalMs": 1454.5999999046326
1288
+ },
1289
+ {
1290
+ "loadMs": 0,
1291
+ "inferenceMs": 1462.2999997138977,
1292
+ "disposeMs": 0,
1293
+ "totalMs": 1462.5
1294
+ },
1295
+ {
1296
+ "loadMs": 0,
1297
+ "inferenceMs": 1463,
1298
+ "disposeMs": 0,
1299
+ "totalMs": 1463.1000003814697
1300
+ },
1301
+ {
1302
+ "loadMs": 0,
1303
+ "inferenceMs": 1457,
1304
+ "disposeMs": 0,
1305
+ "totalMs": 1457.0999999046326
1306
+ },
1307
+ {
1308
+ "loadMs": 0,
1309
+ "inferenceMs": 1485.3000001907349,
1310
+ "disposeMs": 0,
1311
+ "totalMs": 1485.4000000953674
1312
+ },
1313
+ {
1314
+ "loadMs": 0,
1315
+ "inferenceMs": 1458.7000002861023,
1316
+ "disposeMs": 0,
1317
+ "totalMs": 1458.9000000953674
1318
+ },
1319
+ {
1320
+ "loadMs": 0,
1321
+ "inferenceMs": 1460.5,
1322
+ "disposeMs": 0,
1323
+ "totalMs": 1460.5
1324
+ }
1325
+ ],
1326
+ "first": {
1327
+ "loadMs": 624.7999997138977,
1328
+ "inferenceMs": 1500.0999999046326,
1329
+ "disposeMs": 0,
1330
+ "totalMs": 2128.5
1331
+ }
1332
+ },
1333
+ {
1334
+ "size": "150m",
1335
+ "device": "cpu",
1336
+ "dtype": "fp32",
1337
+ "samples": [
1338
+ {
1339
+ "loadMs": 0,
1340
+ "inferenceMs": 1294.0999999046326,
1341
+ "disposeMs": 0,
1342
+ "totalMs": 1294.0999999046326
1343
+ },
1344
+ {
1345
+ "loadMs": 0,
1346
+ "inferenceMs": 1296.7000002861023,
1347
+ "disposeMs": 0,
1348
+ "totalMs": 1296.9000000953674
1349
+ },
1350
+ {
1351
+ "loadMs": 0,
1352
+ "inferenceMs": 1294.7999997138977,
1353
+ "disposeMs": 0,
1354
+ "totalMs": 1294.9000000953674
1355
+ },
1356
+ {
1357
+ "loadMs": 0,
1358
+ "inferenceMs": 1306.6999998092651,
1359
+ "disposeMs": 0,
1360
+ "totalMs": 1306.8999996185303
1361
+ },
1362
+ {
1363
+ "loadMs": 0,
1364
+ "inferenceMs": 1316.1999998092651,
1365
+ "disposeMs": 0,
1366
+ "totalMs": 1316.1999998092651
1367
+ },
1368
+ {
1369
+ "loadMs": 0,
1370
+ "inferenceMs": 1301.5,
1371
+ "disposeMs": 0,
1372
+ "totalMs": 1301.5999999046326
1373
+ },
1374
+ {
1375
+ "loadMs": 0,
1376
+ "inferenceMs": 1471,
1377
+ "disposeMs": 0,
1378
+ "totalMs": 1471.0999999046326
1379
+ },
1380
+ {
1381
+ "loadMs": 0,
1382
+ "inferenceMs": 1334.0999999046326,
1383
+ "disposeMs": 0,
1384
+ "totalMs": 1334.2999997138977
1385
+ },
1386
+ {
1387
+ "loadMs": 0,
1388
+ "inferenceMs": 1319.1999998092651,
1389
+ "disposeMs": 0,
1390
+ "totalMs": 1319.4000000953674
1391
+ },
1392
+ {
1393
+ "loadMs": 0,
1394
+ "inferenceMs": 1307.5999999046326,
1395
+ "disposeMs": 0,
1396
+ "totalMs": 1307.6999998092651
1397
+ },
1398
+ {
1399
+ "loadMs": 0,
1400
+ "inferenceMs": 1308,
1401
+ "disposeMs": 0,
1402
+ "totalMs": 1308.0999999046326
1403
+ },
1404
+ {
1405
+ "loadMs": 0,
1406
+ "inferenceMs": 1368.1000003814697,
1407
+ "disposeMs": 0,
1408
+ "totalMs": 1368.2000002861023
1409
+ },
1410
+ {
1411
+ "loadMs": 0,
1412
+ "inferenceMs": 1294.7999997138977,
1413
+ "disposeMs": 0,
1414
+ "totalMs": 1295
1415
+ },
1416
+ {
1417
+ "loadMs": 0,
1418
+ "inferenceMs": 1294.5,
1419
+ "disposeMs": 0,
1420
+ "totalMs": 1294.5
1421
+ },
1422
+ {
1423
+ "loadMs": 0,
1424
+ "inferenceMs": 1301.2999997138977,
1425
+ "disposeMs": 0,
1426
+ "totalMs": 1301.5
1427
+ },
1428
+ {
1429
+ "loadMs": 0,
1430
+ "inferenceMs": 1295.5,
1431
+ "disposeMs": 0,
1432
+ "totalMs": 1295.5999999046326
1433
+ },
1434
+ {
1435
+ "loadMs": 0,
1436
+ "inferenceMs": 1297.5999999046326,
1437
+ "disposeMs": 0,
1438
+ "totalMs": 1297.6999998092651
1439
+ },
1440
+ {
1441
+ "loadMs": 0,
1442
+ "inferenceMs": 1296.1000003814697,
1443
+ "disposeMs": 0,
1444
+ "totalMs": 1296.1000003814697
1445
+ },
1446
+ {
1447
+ "loadMs": 0,
1448
+ "inferenceMs": 1302.4000000953674,
1449
+ "disposeMs": 0,
1450
+ "totalMs": 1302.5
1451
+ },
1452
+ {
1453
+ "loadMs": 0,
1454
+ "inferenceMs": 1311.0999999046326,
1455
+ "disposeMs": 0,
1456
+ "totalMs": 1311.0999999046326
1457
+ }
1458
+ ],
1459
+ "first": {
1460
+ "loadMs": 1332.7999997138977,
1461
+ "inferenceMs": 1320.2000002861023,
1462
+ "disposeMs": 0,
1463
+ "totalMs": 2657.5
1464
+ }
1465
+ },
1466
+ {
1467
+ "size": "150m",
1468
+ "device": "gpu",
1469
+ "dtype": "fp32",
1470
+ "samples": [
1471
+ {
1472
+ "loadMs": 0,
1473
+ "inferenceMs": 68.40000009536743,
1474
+ "disposeMs": 0,
1475
+ "totalMs": 68.5
1476
+ },
1477
+ {
1478
+ "loadMs": 0,
1479
+ "inferenceMs": 67.59999990463257,
1480
+ "disposeMs": 0,
1481
+ "totalMs": 67.59999990463257
1482
+ },
1483
+ {
1484
+ "loadMs": 0,
1485
+ "inferenceMs": 67.40000009536743,
1486
+ "disposeMs": 0,
1487
+ "totalMs": 67.40000009536743
1488
+ },
1489
+ {
1490
+ "loadMs": 0,
1491
+ "inferenceMs": 68.5,
1492
+ "disposeMs": 0,
1493
+ "totalMs": 68.59999990463257
1494
+ },
1495
+ {
1496
+ "loadMs": 0,
1497
+ "inferenceMs": 68.69999980926514,
1498
+ "disposeMs": 0,
1499
+ "totalMs": 68.69999980926514
1500
+ },
1501
+ {
1502
+ "loadMs": 0,
1503
+ "inferenceMs": 66.90000009536743,
1504
+ "disposeMs": 0,
1505
+ "totalMs": 67.10000038146973
1506
+ },
1507
+ {
1508
+ "loadMs": 0,
1509
+ "inferenceMs": 69,
1510
+ "disposeMs": 0,
1511
+ "totalMs": 69.10000038146973
1512
+ },
1513
+ {
1514
+ "loadMs": 0,
1515
+ "inferenceMs": 68.30000019073486,
1516
+ "disposeMs": 0,
1517
+ "totalMs": 68.30000019073486
1518
+ },
1519
+ {
1520
+ "loadMs": 0,
1521
+ "inferenceMs": 68,
1522
+ "disposeMs": 0,
1523
+ "totalMs": 68
1524
+ },
1525
+ {
1526
+ "loadMs": 0,
1527
+ "inferenceMs": 66.90000009536743,
1528
+ "disposeMs": 0,
1529
+ "totalMs": 67
1530
+ },
1531
+ {
1532
+ "loadMs": 0,
1533
+ "inferenceMs": 68.30000019073486,
1534
+ "disposeMs": 0,
1535
+ "totalMs": 68.30000019073486
1536
+ },
1537
+ {
1538
+ "loadMs": 0,
1539
+ "inferenceMs": 67.69999980926514,
1540
+ "disposeMs": 0,
1541
+ "totalMs": 67.69999980926514
1542
+ },
1543
+ {
1544
+ "loadMs": 0,
1545
+ "inferenceMs": 70.40000009536743,
1546
+ "disposeMs": 0,
1547
+ "totalMs": 70.40000009536743
1548
+ },
1549
+ {
1550
+ "loadMs": 0,
1551
+ "inferenceMs": 68.19999980926514,
1552
+ "disposeMs": 0,
1553
+ "totalMs": 68.19999980926514
1554
+ },
1555
+ {
1556
+ "loadMs": 0,
1557
+ "inferenceMs": 69.80000019073486,
1558
+ "disposeMs": 0,
1559
+ "totalMs": 69.90000009536743
1560
+ },
1561
+ {
1562
+ "loadMs": 0,
1563
+ "inferenceMs": 69.19999980926514,
1564
+ "disposeMs": 0,
1565
+ "totalMs": 69.40000009536743
1566
+ },
1567
+ {
1568
+ "loadMs": 0,
1569
+ "inferenceMs": 69.7000002861023,
1570
+ "disposeMs": 0,
1571
+ "totalMs": 69.7000002861023
1572
+ },
1573
+ {
1574
+ "loadMs": 0,
1575
+ "inferenceMs": 70.09999990463257,
1576
+ "disposeMs": 0,
1577
+ "totalMs": 70.19999980926514
1578
+ },
1579
+ {
1580
+ "loadMs": 0,
1581
+ "inferenceMs": 68.40000009536743,
1582
+ "disposeMs": 0,
1583
+ "totalMs": 68.5
1584
+ },
1585
+ {
1586
+ "loadMs": 0,
1587
+ "inferenceMs": 68.60000038146973,
1588
+ "disposeMs": 0,
1589
+ "totalMs": 68.7000002861023
1590
+ }
1591
+ ],
1592
+ "first": {
1593
+ "loadMs": 1583.8999996185303,
1594
+ "inferenceMs": 199.80000019073486,
1595
+ "disposeMs": 0,
1596
+ "totalMs": 1787.5
1597
+ }
1598
+ },
1599
+ {
1600
+ "size": "400m",
1601
+ "device": "cpu",
1602
+ "dtype": "q8",
1603
+ "samples": [
1604
+ {
1605
+ "loadMs": 0,
1606
+ "inferenceMs": 4459.099999904633,
1607
+ "disposeMs": 0,
1608
+ "totalMs": 4459.300000190735
1609
+ },
1610
+ {
1611
+ "loadMs": 0,
1612
+ "inferenceMs": 4401.599999904633,
1613
+ "disposeMs": 0,
1614
+ "totalMs": 4401.699999809265
1615
+ },
1616
+ {
1617
+ "loadMs": 0,
1618
+ "inferenceMs": 4398.39999961853,
1619
+ "disposeMs": 0,
1620
+ "totalMs": 4398.39999961853
1621
+ },
1622
+ {
1623
+ "loadMs": 0,
1624
+ "inferenceMs": 4398.599999904633,
1625
+ "disposeMs": 0,
1626
+ "totalMs": 4398.800000190735
1627
+ },
1628
+ {
1629
+ "loadMs": 0,
1630
+ "inferenceMs": 4435.300000190735,
1631
+ "disposeMs": 0,
1632
+ "totalMs": 4435.300000190735
1633
+ },
1634
+ {
1635
+ "loadMs": 0,
1636
+ "inferenceMs": 4398.5,
1637
+ "disposeMs": 0,
1638
+ "totalMs": 4398.599999904633
1639
+ },
1640
+ {
1641
+ "loadMs": 0,
1642
+ "inferenceMs": 4395.5,
1643
+ "disposeMs": 0,
1644
+ "totalMs": 4395.5
1645
+ },
1646
+ {
1647
+ "loadMs": 0,
1648
+ "inferenceMs": 4555.099999904633,
1649
+ "disposeMs": 0,
1650
+ "totalMs": 4555.199999809265
1651
+ },
1652
+ {
1653
+ "loadMs": 0,
1654
+ "inferenceMs": 4439.900000095367,
1655
+ "disposeMs": 0,
1656
+ "totalMs": 4440.099999904633
1657
+ },
1658
+ {
1659
+ "loadMs": 0,
1660
+ "inferenceMs": 4516.39999961853,
1661
+ "disposeMs": 0,
1662
+ "totalMs": 4516.39999961853
1663
+ },
1664
+ {
1665
+ "loadMs": 0,
1666
+ "inferenceMs": 4517.800000190735,
1667
+ "disposeMs": 0,
1668
+ "totalMs": 4517.900000095367
1669
+ },
1670
+ {
1671
+ "loadMs": 0,
1672
+ "inferenceMs": 4384.5,
1673
+ "disposeMs": 0,
1674
+ "totalMs": 4384.599999904633
1675
+ },
1676
+ {
1677
+ "loadMs": 0,
1678
+ "inferenceMs": 4414.099999904633,
1679
+ "disposeMs": 0,
1680
+ "totalMs": 4414.199999809265
1681
+ },
1682
+ {
1683
+ "loadMs": 0,
1684
+ "inferenceMs": 4423.5,
1685
+ "disposeMs": 0,
1686
+ "totalMs": 4423.599999904633
1687
+ },
1688
+ {
1689
+ "loadMs": 0,
1690
+ "inferenceMs": 4411.400000095367,
1691
+ "disposeMs": 0,
1692
+ "totalMs": 4411.599999904633
1693
+ },
1694
+ {
1695
+ "loadMs": 0,
1696
+ "inferenceMs": 4388.800000190735,
1697
+ "disposeMs": 0,
1698
+ "totalMs": 4388.800000190735
1699
+ },
1700
+ {
1701
+ "loadMs": 0,
1702
+ "inferenceMs": 4455.800000190735,
1703
+ "disposeMs": 0,
1704
+ "totalMs": 4455.900000095367
1705
+ },
1706
+ {
1707
+ "loadMs": 0,
1708
+ "inferenceMs": 4386.599999904633,
1709
+ "disposeMs": 0,
1710
+ "totalMs": 4386.700000286102
1711
+ },
1712
+ {
1713
+ "loadMs": 0,
1714
+ "inferenceMs": 4396.89999961853,
1715
+ "disposeMs": 0,
1716
+ "totalMs": 4397.099999904633
1717
+ },
1718
+ {
1719
+ "loadMs": 0,
1720
+ "inferenceMs": 4384.699999809265,
1721
+ "disposeMs": 0,
1722
+ "totalMs": 4384.699999809265
1723
+ }
1724
+ ],
1725
+ "first": {
1726
+ "loadMs": 1267.7999997138977,
1727
+ "inferenceMs": 4494.200000286102,
1728
+ "disposeMs": 0,
1729
+ "totalMs": 5766
1730
+ }
1731
+ },
1732
+ {
1733
+ "size": "400m",
1734
+ "device": "cpu",
1735
+ "dtype": "fp32",
1736
+ "samples": [
1737
+ {
1738
+ "loadMs": 0,
1739
+ "inferenceMs": 3903.3999996185303,
1740
+ "disposeMs": 0,
1741
+ "totalMs": 3903.5
1742
+ },
1743
+ {
1744
+ "loadMs": 0,
1745
+ "inferenceMs": 3902.2999997138977,
1746
+ "disposeMs": 0,
1747
+ "totalMs": 3902.6000003814697
1748
+ },
1749
+ {
1750
+ "loadMs": 0,
1751
+ "inferenceMs": 3952.4000000953674,
1752
+ "disposeMs": 0,
1753
+ "totalMs": 3952.699999809265
1754
+ },
1755
+ {
1756
+ "loadMs": 0,
1757
+ "inferenceMs": 4021.300000190735,
1758
+ "disposeMs": 0,
1759
+ "totalMs": 4021.5
1760
+ },
1761
+ {
1762
+ "loadMs": 0,
1763
+ "inferenceMs": 3898.199999809265,
1764
+ "disposeMs": 0,
1765
+ "totalMs": 3898.4000000953674
1766
+ },
1767
+ {
1768
+ "loadMs": 0,
1769
+ "inferenceMs": 3897.9000000953674,
1770
+ "disposeMs": 0,
1771
+ "totalMs": 3898
1772
+ },
1773
+ {
1774
+ "loadMs": 0,
1775
+ "inferenceMs": 3891.800000190735,
1776
+ "disposeMs": 0,
1777
+ "totalMs": 3891.9000000953674
1778
+ },
1779
+ {
1780
+ "loadMs": 0,
1781
+ "inferenceMs": 3899.800000190735,
1782
+ "disposeMs": 0,
1783
+ "totalMs": 3900
1784
+ },
1785
+ {
1786
+ "loadMs": 0,
1787
+ "inferenceMs": 3902.5,
1788
+ "disposeMs": 0,
1789
+ "totalMs": 3902.5
1790
+ },
1791
+ {
1792
+ "loadMs": 0,
1793
+ "inferenceMs": 3919.8999996185303,
1794
+ "disposeMs": 0,
1795
+ "totalMs": 3920
1796
+ },
1797
+ {
1798
+ "loadMs": 0,
1799
+ "inferenceMs": 3939.800000190735,
1800
+ "disposeMs": 0,
1801
+ "totalMs": 3939.800000190735
1802
+ },
1803
+ {
1804
+ "loadMs": 0,
1805
+ "inferenceMs": 3995.7999997138977,
1806
+ "disposeMs": 0,
1807
+ "totalMs": 3995.9000000953674
1808
+ },
1809
+ {
1810
+ "loadMs": 0,
1811
+ "inferenceMs": 4017,
1812
+ "disposeMs": 0,
1813
+ "totalMs": 4017.2000002861023
1814
+ },
1815
+ {
1816
+ "loadMs": 0,
1817
+ "inferenceMs": 3927.699999809265,
1818
+ "disposeMs": 0,
1819
+ "totalMs": 3927.9000000953674
1820
+ },
1821
+ {
1822
+ "loadMs": 0,
1823
+ "inferenceMs": 3903.4000000953674,
1824
+ "disposeMs": 0,
1825
+ "totalMs": 3903.5999999046326
1826
+ },
1827
+ {
1828
+ "loadMs": 0,
1829
+ "inferenceMs": 3922.6000003814697,
1830
+ "disposeMs": 0,
1831
+ "totalMs": 3922.7000002861023
1832
+ },
1833
+ {
1834
+ "loadMs": 0,
1835
+ "inferenceMs": 3900.0999999046326,
1836
+ "disposeMs": 0,
1837
+ "totalMs": 3900.0999999046326
1838
+ },
1839
+ {
1840
+ "loadMs": 0,
1841
+ "inferenceMs": 3932.4000000953674,
1842
+ "disposeMs": 0,
1843
+ "totalMs": 3932.5
1844
+ },
1845
+ {
1846
+ "loadMs": 0,
1847
+ "inferenceMs": 3901.2000002861023,
1848
+ "disposeMs": 0,
1849
+ "totalMs": 3901.300000190735
1850
+ },
1851
+ {
1852
+ "loadMs": 0,
1853
+ "inferenceMs": 3896.2999997138977,
1854
+ "disposeMs": 0,
1855
+ "totalMs": 3896.5
1856
+ }
1857
+ ],
1858
+ "first": {
1859
+ "loadMs": 4731.599999904633,
1860
+ "inferenceMs": 4044.300000190735,
1861
+ "disposeMs": 0,
1862
+ "totalMs": 8783.199999809265
1863
+ }
1864
+ },
1865
+ {
1866
+ "size": "400m",
1867
+ "device": "gpu",
1868
+ "dtype": "fp32",
1869
+ "samples": [
1870
+ {
1871
+ "loadMs": 0,
1872
+ "inferenceMs": 196.30000019073486,
1873
+ "disposeMs": 0,
1874
+ "totalMs": 196.30000019073486
1875
+ },
1876
+ {
1877
+ "loadMs": 0,
1878
+ "inferenceMs": 196.7000002861023,
1879
+ "disposeMs": 0,
1880
+ "totalMs": 196.7000002861023
1881
+ },
1882
+ {
1883
+ "loadMs": 0,
1884
+ "inferenceMs": 192.09999990463257,
1885
+ "disposeMs": 0,
1886
+ "totalMs": 192.09999990463257
1887
+ },
1888
+ {
1889
+ "loadMs": 0,
1890
+ "inferenceMs": 199.89999961853027,
1891
+ "disposeMs": 0,
1892
+ "totalMs": 200
1893
+ },
1894
+ {
1895
+ "loadMs": 0,
1896
+ "inferenceMs": 197.7999997138977,
1897
+ "disposeMs": 0,
1898
+ "totalMs": 198
1899
+ },
1900
+ {
1901
+ "loadMs": 0,
1902
+ "inferenceMs": 196.40000009536743,
1903
+ "disposeMs": 0,
1904
+ "totalMs": 196.5
1905
+ },
1906
+ {
1907
+ "loadMs": 0,
1908
+ "inferenceMs": 190.90000009536743,
1909
+ "disposeMs": 0,
1910
+ "totalMs": 191
1911
+ },
1912
+ {
1913
+ "loadMs": 0,
1914
+ "inferenceMs": 195.40000009536743,
1915
+ "disposeMs": 0,
1916
+ "totalMs": 195.40000009536743
1917
+ },
1918
+ {
1919
+ "loadMs": 0,
1920
+ "inferenceMs": 192.40000009536743,
1921
+ "disposeMs": 0,
1922
+ "totalMs": 192.5
1923
+ },
1924
+ {
1925
+ "loadMs": 0,
1926
+ "inferenceMs": 193.69999980926514,
1927
+ "disposeMs": 0,
1928
+ "totalMs": 193.80000019073486
1929
+ },
1930
+ {
1931
+ "loadMs": 0,
1932
+ "inferenceMs": 191.09999990463257,
1933
+ "disposeMs": 0,
1934
+ "totalMs": 191.19999980926514
1935
+ },
1936
+ {
1937
+ "loadMs": 0,
1938
+ "inferenceMs": 190.90000009536743,
1939
+ "disposeMs": 0,
1940
+ "totalMs": 191.09999990463257
1941
+ },
1942
+ {
1943
+ "loadMs": 0,
1944
+ "inferenceMs": 190.59999990463257,
1945
+ "disposeMs": 0,
1946
+ "totalMs": 190.59999990463257
1947
+ },
1948
+ {
1949
+ "loadMs": 0,
1950
+ "inferenceMs": 190.09999990463257,
1951
+ "disposeMs": 0,
1952
+ "totalMs": 190.09999990463257
1953
+ },
1954
+ {
1955
+ "loadMs": 0,
1956
+ "inferenceMs": 189.59999990463257,
1957
+ "disposeMs": 0,
1958
+ "totalMs": 189.59999990463257
1959
+ },
1960
+ {
1961
+ "loadMs": 0,
1962
+ "inferenceMs": 192.30000019073486,
1963
+ "disposeMs": 0,
1964
+ "totalMs": 192.30000019073486
1965
+ },
1966
+ {
1967
+ "loadMs": 0,
1968
+ "inferenceMs": 191.90000009536743,
1969
+ "disposeMs": 0,
1970
+ "totalMs": 192.09999990463257
1971
+ },
1972
+ {
1973
+ "loadMs": 0,
1974
+ "inferenceMs": 189.40000009536743,
1975
+ "disposeMs": 0,
1976
+ "totalMs": 189.5
1977
+ },
1978
+ {
1979
+ "loadMs": 0,
1980
+ "inferenceMs": 191.5,
1981
+ "disposeMs": 0,
1982
+ "totalMs": 191.5
1983
+ },
1984
+ {
1985
+ "loadMs": 0,
1986
+ "inferenceMs": 190.90000009536743,
1987
+ "disposeMs": 0,
1988
+ "totalMs": 191
1989
+ }
1990
+ ],
1991
+ "first": {
1992
+ "loadMs": 4818.700000286102,
1993
+ "inferenceMs": 1134.8000001907349,
1994
+ "disposeMs": 0,
1995
+ "totalMs": 5959
1996
+ }
1997
+ },
1998
+ {
1999
+ "size": "1b",
2000
+ "device": "cpu",
2001
+ "dtype": "q8",
2002
+ "samples": [
2003
+ {
2004
+ "loadMs": 0,
2005
+ "inferenceMs": 12250.199999809265,
2006
+ "disposeMs": 0,
2007
+ "totalMs": 12250.299999713898
2008
+ },
2009
+ {
2010
+ "loadMs": 0,
2011
+ "inferenceMs": 12276.099999904633,
2012
+ "disposeMs": 0,
2013
+ "totalMs": 12276.199999809265
2014
+ },
2015
+ {
2016
+ "loadMs": 0,
2017
+ "inferenceMs": 12321.10000038147,
2018
+ "disposeMs": 0,
2019
+ "totalMs": 12321.200000286102
2020
+ },
2021
+ {
2022
+ "loadMs": 0,
2023
+ "inferenceMs": 12460.5,
2024
+ "disposeMs": 0,
2025
+ "totalMs": 12460.599999904633
2026
+ },
2027
+ {
2028
+ "loadMs": 0,
2029
+ "inferenceMs": 12095.800000190735,
2030
+ "disposeMs": 0,
2031
+ "totalMs": 12096.099999904633
2032
+ },
2033
+ {
2034
+ "loadMs": 0,
2035
+ "inferenceMs": 12153.699999809265,
2036
+ "disposeMs": 0,
2037
+ "totalMs": 12153.699999809265
2038
+ },
2039
+ {
2040
+ "loadMs": 0,
2041
+ "inferenceMs": 12122.599999904633,
2042
+ "disposeMs": 0,
2043
+ "totalMs": 12122.699999809265
2044
+ },
2045
+ {
2046
+ "loadMs": 0,
2047
+ "inferenceMs": 13067.200000286102,
2048
+ "disposeMs": 0,
2049
+ "totalMs": 13067.5
2050
+ },
2051
+ {
2052
+ "loadMs": 0,
2053
+ "inferenceMs": 12957.099999904633,
2054
+ "disposeMs": 0,
2055
+ "totalMs": 12957.699999809265
2056
+ },
2057
+ {
2058
+ "loadMs": 0,
2059
+ "inferenceMs": 12998.900000095367,
2060
+ "disposeMs": 0,
2061
+ "totalMs": 12999.700000286102
2062
+ },
2063
+ {
2064
+ "loadMs": 0,
2065
+ "inferenceMs": 12278.200000286102,
2066
+ "disposeMs": 0,
2067
+ "totalMs": 12278.400000095367
2068
+ },
2069
+ {
2070
+ "loadMs": 0,
2071
+ "inferenceMs": 12153.900000095367,
2072
+ "disposeMs": 0,
2073
+ "totalMs": 12153.900000095367
2074
+ },
2075
+ {
2076
+ "loadMs": 0,
2077
+ "inferenceMs": 12190.800000190735,
2078
+ "disposeMs": 0,
2079
+ "totalMs": 12191.099999904633
2080
+ },
2081
+ {
2082
+ "loadMs": 0,
2083
+ "inferenceMs": 13207.400000095367,
2084
+ "disposeMs": 0,
2085
+ "totalMs": 13207.799999713898
2086
+ },
2087
+ {
2088
+ "loadMs": 0,
2089
+ "inferenceMs": 12638,
2090
+ "disposeMs": 0,
2091
+ "totalMs": 12638.300000190735
2092
+ },
2093
+ {
2094
+ "loadMs": 0,
2095
+ "inferenceMs": 12439.400000095367,
2096
+ "disposeMs": 0,
2097
+ "totalMs": 12439.599999904633
2098
+ },
2099
+ {
2100
+ "loadMs": 0,
2101
+ "inferenceMs": 12800.800000190735,
2102
+ "disposeMs": 0,
2103
+ "totalMs": 12802
2104
+ },
2105
+ {
2106
+ "loadMs": 0,
2107
+ "inferenceMs": 12897.5,
2108
+ "disposeMs": 0,
2109
+ "totalMs": 12897.799999713898
2110
+ },
2111
+ {
2112
+ "loadMs": 0,
2113
+ "inferenceMs": 13246.199999809265,
2114
+ "disposeMs": 0,
2115
+ "totalMs": 13247
2116
+ },
2117
+ {
2118
+ "loadMs": 0,
2119
+ "inferenceMs": 12444.599999904633,
2120
+ "disposeMs": 0,
2121
+ "totalMs": 12445.599999904633
2122
+ }
2123
+ ],
2124
+ "first": {
2125
+ "loadMs": 3724.5999999046326,
2126
+ "inferenceMs": 12304.200000286102,
2127
+ "disposeMs": 0,
2128
+ "totalMs": 16034.699999809265
2129
+ }
2130
+ },
2131
+ {
2132
+ "size": "1b",
2133
+ "device": "gpu",
2134
+ "dtype": "fp32",
2135
+ "samples": [
2136
+ {
2137
+ "loadMs": 0,
2138
+ "inferenceMs": 502.80000019073486,
2139
+ "disposeMs": 0,
2140
+ "totalMs": 502.80000019073486
2141
+ },
2142
+ {
2143
+ "loadMs": 0,
2144
+ "inferenceMs": 484.5,
2145
+ "disposeMs": 0,
2146
+ "totalMs": 484.59999990463257
2147
+ },
2148
+ {
2149
+ "loadMs": 0,
2150
+ "inferenceMs": 481.3999996185303,
2151
+ "disposeMs": 0,
2152
+ "totalMs": 481.59999990463257
2153
+ },
2154
+ {
2155
+ "loadMs": 0,
2156
+ "inferenceMs": 458.5,
2157
+ "disposeMs": 0,
2158
+ "totalMs": 458.59999990463257
2159
+ },
2160
+ {
2161
+ "loadMs": 0,
2162
+ "inferenceMs": 492.69999980926514,
2163
+ "disposeMs": 0,
2164
+ "totalMs": 492.7999997138977
2165
+ },
2166
+ {
2167
+ "loadMs": 0,
2168
+ "inferenceMs": 481.90000009536743,
2169
+ "disposeMs": 0,
2170
+ "totalMs": 482.09999990463257
2171
+ },
2172
+ {
2173
+ "loadMs": 0,
2174
+ "inferenceMs": 486,
2175
+ "disposeMs": 0,
2176
+ "totalMs": 486
2177
+ },
2178
+ {
2179
+ "loadMs": 0,
2180
+ "inferenceMs": 451.40000009536743,
2181
+ "disposeMs": 0,
2182
+ "totalMs": 451.59999990463257
2183
+ },
2184
+ {
2185
+ "loadMs": 0,
2186
+ "inferenceMs": 487.59999990463257,
2187
+ "disposeMs": 0,
2188
+ "totalMs": 487.69999980926514
2189
+ },
2190
+ {
2191
+ "loadMs": 0,
2192
+ "inferenceMs": 488.2000002861023,
2193
+ "disposeMs": 0,
2194
+ "totalMs": 488.30000019073486
2195
+ },
2196
+ {
2197
+ "loadMs": 0,
2198
+ "inferenceMs": 489.5,
2199
+ "disposeMs": 0,
2200
+ "totalMs": 489.5
2201
+ },
2202
+ {
2203
+ "loadMs": 0,
2204
+ "inferenceMs": 457.90000009536743,
2205
+ "disposeMs": 0,
2206
+ "totalMs": 457.90000009536743
2207
+ },
2208
+ {
2209
+ "loadMs": 0,
2210
+ "inferenceMs": 485.40000009536743,
2211
+ "disposeMs": 0,
2212
+ "totalMs": 485.5
2213
+ },
2214
+ {
2215
+ "loadMs": 0,
2216
+ "inferenceMs": 489.80000019073486,
2217
+ "disposeMs": 0,
2218
+ "totalMs": 489.90000009536743
2219
+ },
2220
+ {
2221
+ "loadMs": 0,
2222
+ "inferenceMs": 497.09999990463257,
2223
+ "disposeMs": 0,
2224
+ "totalMs": 497.09999990463257
2225
+ },
2226
+ {
2227
+ "loadMs": 0,
2228
+ "inferenceMs": 463.5,
2229
+ "disposeMs": 0,
2230
+ "totalMs": 463.59999990463257
2231
+ },
2232
+ {
2233
+ "loadMs": 0,
2234
+ "inferenceMs": 504.30000019073486,
2235
+ "disposeMs": 0,
2236
+ "totalMs": 504.30000019073486
2237
+ },
2238
+ {
2239
+ "loadMs": 0,
2240
+ "inferenceMs": 515.1999998092651,
2241
+ "disposeMs": 0,
2242
+ "totalMs": 515.4000000953674
2243
+ },
2244
+ {
2245
+ "loadMs": 0,
2246
+ "inferenceMs": 496.59999990463257,
2247
+ "disposeMs": 0,
2248
+ "totalMs": 496.59999990463257
2249
+ },
2250
+ {
2251
+ "loadMs": 0,
2252
+ "inferenceMs": 464.69999980926514,
2253
+ "disposeMs": 0,
2254
+ "totalMs": 464.69999980926514
2255
+ }
2256
+ ],
2257
+ "first": {
2258
+ "loadMs": 6311.900000095367,
2259
+ "inferenceMs": 3105.5999999046326,
2260
+ "disposeMs": 0,
2261
+ "totalMs": 9436
2262
+ }
2263
+ }
2264
+ ],
2265
+ "status": "complete",
2266
+ "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/154.0.0.0 Safari/537.36",
2267
+ "threads": 10,
2268
+ "gpu": {
2269
+ "vendor": "apple",
2270
+ "architecture": "metal-3",
2271
+ "device": "",
2272
+ "description": ""
2273
+ },
2274
+ "input": {
2275
+ "state": {
2276
+ "message": "My card was charged twice for the same order. Please refund the duplicate charge.",
2277
+ "order": {
2278
+ "status": "paid",
2279
+ "paymentCount": 2
2280
+ }
2281
+ },
2282
+ "decisions": [
2283
+ {
2284
+ "id": "intent",
2285
+ "instruction": "Identify the primary intent of the customer.",
2286
+ "type": "choice",
2287
+ "options": [
2288
+ {
2289
+ "value": "refund",
2290
+ "description": "The customer wants to recover money charged."
2291
+ },
2292
+ {
2293
+ "value": "cancel",
2294
+ "description": "The customer wants to cancel an order."
2295
+ },
2296
+ {
2297
+ "value": "other",
2298
+ "description": "The request does not match the other intents."
2299
+ }
2300
+ ]
2301
+ },
2302
+ {
2303
+ "id": "duplicateCharge",
2304
+ "instruction": "The customer was charged twice for the same order.",
2305
+ "type": "noul",
2306
+ "outcomes": {
2307
+ "true": {
2308
+ "description": "The same order was charged more than once."
2309
+ },
2310
+ "false": {
2311
+ "description": "There was one charge or the charges were for different orders."
2312
+ }
2313
+ }
2314
+ },
2315
+ {
2316
+ "id": "urgency",
2317
+ "instruction": "Rate the urgency of this support request using the ordered levels.",
2318
+ "type": "score",
2319
+ "levels": [
2320
+ {
2321
+ "value": "0",
2322
+ "description": "An informational question without loss or blocked work."
2323
+ },
2324
+ {
2325
+ "value": "1",
2326
+ "description": "A limited loss or a problem with a workaround."
2327
+ },
2328
+ {
2329
+ "value": "2",
2330
+ "description": "Major loss or blocked work requiring immediate action."
2331
+ }
2332
+ ]
2333
+ }
2334
+ ]
2335
+ },
2336
+ "crossOriginIsolated": false,
2337
+ "environment": {
2338
+ "machine": "Apple M4",
2339
+ "memoryGiB": 24,
2340
+ "os": "macOS 26.6.2",
2341
+ "browser": "Headless Chromium 154, Metal WebGPU",
2342
+ "bun": "1.4.0",
2343
+ "onnxruntimeWeb": "1.30.0",
2344
+ "transformersJs": "3.8.1",
2345
+ "assetRevision": "4987a6677e5810585d0d6068f4598934848838d5",
2346
+ "sourceRevision": "bd8c5eec552463ed815a9daf66356816bf7b24c2",
2347
+ "date": "2026-10-19",
2348
+ "cache": "Exact published assets preloaded in transformers-cache; no weight download in measured setup",
2349
+ "conditions": "Local development session; power mode and thermals were not controlled"
2350
+ }
2351
+ }
scripts/benchmark/playground.js CHANGED
@@ -3,7 +3,8 @@ window.crivoBenchmark = {
3
  runs: [],
4
  status: 'starting',
5
  userAgent: navigator.userAgent,
6
- threads: navigator.hardwareConcurrency
 
7
  }
8
  ;(async () => {
9
  const report = window.crivoBenchmark
 
3
  runs: [],
4
  status: 'starting',
5
  userAgent: navigator.userAgent,
6
+ threads: navigator.hardwareConcurrency,
7
+ crossOriginIsolated: window.crossOriginIsolated
8
  }
9
  ;(async () => {
10
  const report = window.crivoBenchmark