Versión 1/4
-
Siguiente » -
Versión actual
Emilio Penna, Jueves, 29 de Octubre de 2026 00:14:15 -0300
Shibboleth IdP 5.2.3 load test results (October 2026)¶
Test date: 28/10/2026. Universidad de la República (UdelaR), Uruguay — test IdP.
Script and instructions: LoadTestScript. Previous test on the same VM with IdP 4.3.3: Perf2025.
Environment¶
| Component | Details |
|---|---|
| IdP host | VMware VM, 4 vCPU (Intel Xeon Gold 6138 @ 2.0 GHz), 4 GB RAM, Debian 13 (kernel 6.12) |
| IdP | Shibboleth IdP 5.2.3, Jetty 12.1.12 (net.shibboleth.idp.plugin.jetty 1.0.1) |
| Java | Amazon Corretto 17.0.21, -Xmx1500m -XX:+UseG1GC |
| Request path | JMeter → front-end reverse proxy (Apache httpd 2.2, separate host) → Apache httpd with mod_proxy on the IdP host → Jetty |
| Directory | OpenLDAP on a separate host: search + bind + attribute lookup on every login, no resolver caching |
| IdP configuration | password authentication, client-side session storage, computed persistent NameID, signed assertion (RSA, rsa-sha512), assertion not encrypted, default logging |
| Client | JMeter 5.6.3, non-GUI, Ubuntu 24.04, 8 logical CPUs, 32 GB RAM, same LAN |
Method¶
- SAML2 unsolicited SSO, one complete password login per iteration (2 HTTP requests, see LoadTestScript).
- Three runs of 5 minutes each with 10, 50 and 25 threads, default ramp-up (10 s), no think time.
- IdP restarted before the first run. Figures exclude the first minute of each run (JIT warm-up is visible in the CPU graph).
- A single test account was used for all logins. No caching is configured in the IdP, so every login still queries LDAP, but directory-side caching is favourable.
- Server metrics collected with nmon every 5 s on the IdP host.
Results¶
| Threads | Logins/s | GET avg / p95 (ms) | POST avg / p95 (ms) | Full login avg (ms) | Errors | IdP host CPU | CPU time per login |
|---|---|---|---|---|---|---|---|
| 10 | 123 | 36 / 53 | 44 / 78 | 80 | 0 | 70% | ~23 ms |
| 25 | 172 | 50 / 82 | 94 / 171 | 144 | 0 | 88% | ~21 ms |
| 50 | 165 | 89 / 191 | 213 / 387 | 302 | 12 (0.01%) | 90% | ~22 ms |
"CPU time per login" = average CPU utilisation × 4 vCPU / logins per second (includes the local Apache, about 6% of CPU).
Observations¶
- The IdP is CPU bound. Throughput levels off at about 170 logins/s between 10 and 25 threads. Going to 50 threads adds no throughput and only doubles latency.
- Each login costs about 21–23 ms of CPU on this hardware. This figure is the most useful one for sizing other environments.
- Without saturation (10 threads), a full login takes about 80 ms, including the LDAP round trips.
- No I/O wait; network ~3 MB/s outbound (~18 KB per login). No swap was used, and the heap grew towards
-Xmxas expected with G1. - The 12 errors appeared only at 50 threads: 502 Proxy Error / no response. The front-end proxy reused keep-alive connections the backend Apache had already closed (
(70014)End of file found: proxy: error reading status line). No errors were logged by the IdP.
Comparison with IdP 4.3.3 (Perf2025)¶
| 50 threads, 5 min | 2025: IdP 4.3.3 | 2026: IdP 5.2.3 |
|---|---|---|
| OS / Java / container | Debian 11, Java 11, Tomcat 9 (mod_jk) | Debian 13, Java 17, Jetty 12 (mod_proxy) |
| Test accounts | 1,500 | 1 |
| Logins/s | 167 | 165 |
| Full login avg (ms) | 284 | 302 |
| Errors | 0 | 0.01% |
Same VM, same front-end proxy, but the whole stack was upgraded, so the difference cannot be attributed to the IdP version alone. Capacity at saturation is equivalent.