FederationStats
Federation usage statistics
TODO
I am a stub, please fix me! |
Federation visualization project
- source (ruby on rails) https://repo.niif.hu/gitweb/gitweb.cgi?p=federation-stats.git;a=summary
- live demo https://papigw.aai.niif.hu/fedstats
Running the sample project
- Install Ruby
- Install Rails (
gem install rails
) - Setup a
development.sqlite3
database with therake db:setup
command - Fire up
script/server
, it will run the project on localhost:3000
Statistic types
Currently we have the following types of statistics:
- Unique users per day (
USER_COUNT
) - AuthnResponse per day (
AUTH
) - AuthnResponse per service per day (
SSO_TO_SERVICE
)
Log statistics format
The following simple format is used to convey statistics from IdPs to the central module - the white spaces (new lines) are important:
ENTITYID #ENTITYID# APIKEY #API_KEY# DATE yyyy-mm-dd STAT #STAT_ID# xxxx STAT #STAT_ID# yyyy STAT #STAT_ID# ww | #PEER_ENTITY_1# zz | #PEER_ENTITY_2#
The following sample might help understanding the format:
ENTITYID https://idp.niif.hu/idp/shibboleth APIKEY 0123....... DATE 2009-03-18 STAT AUTH 68 logins STAT USER_COUNT 16 unique userids STAT SSO_TO_SERVICE 1 | urn:geant:niif.hu:niifi:sp:register.ca.niif.hu 12 | https://repo.niif.hu/shibboleth 1 | https://sandbox.aai.niif.hu/shibboleth 5 | https://sysmonitor.hbone.hu/shibboleth 10 | https://www.ki.iif.hu/shibboleth 1 | https://noc6.vh.hbone.hu/shibboleth 21 | https://webadmin.iif.hu/shibboleth 3 | https://rrd-ma.perfsonar.vh.hbone.hu/shibboleth 7 | https://ugyeletes.vh.hbone.hu/shibboleth 2 | https://noc.grid.niif.hu/shibboleth 1 | https://wiki.voip.niif.hu/shibboleth 2 | https://netmonitor.hbone.hu/shibboleth 2 | https://idp.sch.bme.hu:443/opensso/sp/test
Running the log statistics collector
This following script can be used the collect statistics from the idp audit logs of Shibboleth 2 IdP. It is based on Peter Scober's audit_r7.py:
#!/bin/bash
FILES="log-directory/idp-audit-*.log"
TARGETDIR="/tmp/stats"
ENTITYID="idp-entity-id"
APIKEY="aaa....."
AUDIT="/path/to/audit_r7.py"
for f in $FILES
do
echo "Processing $f file..."
DATE=${f:10:10}
LOGINS=`$AUDIT -l $f`
USERS=`$AUDIT -u $f`
SERVICES=`$AUDIT -p $f | sed '/^[0-9]/p' -n`
FILENAME="stat-$f"
echo "ENTITYID $ENTITYID
APIKEY $APIKEY
DATE $DATE
STAT AUTH
$LOGINS
STAT USER_COUNT
$USERS
STAT SSO_TO_SERVICE
$SERVICES
" > $TARGETDIR/$FILENAME
done #for files
Feeding the database with the statistics
The federation statistics rails project contains the script/stat_parser/file.rb
command which can process the statistics format and load the data to the database. Note that this script currently contains an absolute path for the script/runner
script, so you must fix this before use.
Using HTTP-Post to feed the database
When deployed, the rails project provides a /import_stats
URL to which one could POST the generated statistics file.
Creating IdPs
You must create records for the IdPs from which the application accepts statistics. This could be done via the /entities/new
URL. The name of the entity must be the entityID, the type must be 'idp'.
API keys
Every IdP record in the database will contain a 40 character length API key.