Versión 1/4 - Siguiente » - Versión actual
Emilio Penna, Jueves, 29 de Octubre de 2026 00:14:15 -0300


Shibboleth IdP 5.2.3 load test results (October 2026)

Test date: 28/10/2026. Universidad de la República (UdelaR), Uruguay — test IdP.
Script and instructions: LoadTestScript. Previous test on the same VM with IdP 4.3.3: Perf2025.

Environment

Component Details
IdP host VMware VM, 4 vCPU (Intel Xeon Gold 6138 @ 2.0 GHz), 4 GB RAM, Debian 13 (kernel 6.12)
IdP Shibboleth IdP 5.2.3, Jetty 12.1.12 (net.shibboleth.idp.plugin.jetty 1.0.1)
Java Amazon Corretto 17.0.21, -Xmx1500m -XX:+UseG1GC
Request path JMeter → front-end reverse proxy (Apache httpd 2.2, separate host) → Apache httpd with mod_proxy on the IdP host → Jetty
Directory OpenLDAP on a separate host: search + bind + attribute lookup on every login, no resolver caching
IdP configuration password authentication, client-side session storage, computed persistent NameID, signed assertion (RSA, rsa-sha512), assertion not encrypted, default logging
Client JMeter 5.6.3, non-GUI, Ubuntu 24.04, 8 logical CPUs, 32 GB RAM, same LAN

Method

  • SAML2 unsolicited SSO, one complete password login per iteration (2 HTTP requests, see LoadTestScript).
  • Three runs of 5 minutes each with 10, 50 and 25 threads, default ramp-up (10 s), no think time.
  • IdP restarted before the first run. Figures exclude the first minute of each run (JIT warm-up is visible in the CPU graph).
  • A single test account was used for all logins. No caching is configured in the IdP, so every login still queries LDAP, but directory-side caching is favourable.
  • Server metrics collected with nmon every 5 s on the IdP host.

Results

Threads Logins/s GET avg / p95 (ms) POST avg / p95 (ms) Full login avg (ms) Errors IdP host CPU CPU time per login
10 123 36 / 53 44 / 78 80 0 70% ~23 ms
25 172 50 / 82 94 / 171 144 0 88% ~21 ms
50 165 89 / 191 213 / 387 302 12 (0.01%) 90% ~22 ms

"CPU time per login" = average CPU utilisation × 4 vCPU / logins per second (includes the local Apache, about 6% of CPU).

Observations

  • The IdP is CPU bound. Throughput levels off at about 170 logins/s between 10 and 25 threads. Going to 50 threads adds no throughput and only doubles latency.
  • Each login costs about 21–23 ms of CPU on this hardware. This figure is the most useful one for sizing other environments.
  • Without saturation (10 threads), a full login takes about 80 ms, including the LDAP round trips.
  • No I/O wait; network ~3 MB/s outbound (~18 KB per login). No swap was used, and the heap grew towards -Xmx as expected with G1.
  • The 12 errors appeared only at 50 threads: 502 Proxy Error / no response. The front-end proxy reused keep-alive connections the backend Apache had already closed ((70014)End of file found: proxy: error reading status line). No errors were logged by the IdP.

Comparison with IdP 4.3.3 (Perf2025)

50 threads, 5 min 2025: IdP 4.3.3 2026: IdP 5.2.3
OS / Java / container Debian 11, Java 11, Tomcat 9 (mod_jk) Debian 13, Java 17, Jetty 12 (mod_proxy)
Test accounts 1,500 1
Logins/s 167 165
Full login avg (ms) 284 302
Errors 0 0.01%

Same VM, same front-end proxy, but the whole stack was upgraded, so the difference cannot be attributed to the IdP version alone. Capacity at saturation is equivalent.

perf2026-cpu.png (58,6 KB) Emilio Penna, Jueves, 29 de Octubre de 2026 00:14:44 -0300