Saturday, April 16, 2016

javax.transaction.Transactional

Suppose we have a managed CDI bean exposing to JSF-EL two methods:

public void methodA(){
    //do some additional stuff
    methodB();
}
@Transactional
public void methodB(){
    //do actual work by several EJB calls
}

At a first glance you could think:

  • If we call methodA(from a JSF view) we would get a transaction started and all work will be done inside this transaction
  • If we call methodA we will get the same result for whatever methodA is doing

Well, while the first assumption works as stated the second is, surprise, wrong. Each individual call to the mentioned EJB will trigger it's own transaction.

This behaviour is due to the following not obvious at first glance fact: JSF will call CDI methods through a proxy that wraps the actual instance and fires interceptors; in our second case interceptors have been already fired before methodA was actually called and no other interceptors (that would trigger transaction management) will be called due to the fact that instance not proxy methodB will be called.

U3VwcG9zZSB3ZSBoYXZlIGEgbWFuYWdlZCBDREkgYmVhbiBleHBvc2luZyB0byBKU0YtRUwgdHdvIG1ldGhvZHM6DQoNCmBgYGphdmENCnB1YmxpYyB2b2lkIG1ldGhvZEEoKXsNCgkvL2RvIHNvbWUgYWRkaXRpb25hbCBzdHVmZg0KCW1ldGhvZEIoKTsNCn0NCkBUcmFuc2FjdGlvbmFsDQpwdWJsaWMgdm9pZCBtZXRob2RCKCl7DQoJLy9kbyBhY3R1YWwgd29yayBieSBzZXZlcmFsIEVKQiBjYWxscw0KfQ0KYGBgDQpBdCBhIGZpcnN0IGdsYW5jZSB5b3UgY291bGQgdGhpbms6DQoqIElmIHdlIGNhbGwgYG1ldGhvZEFgKGZyb20gYSBKU0YgdmlldykgIHdlIHdvdWxkIGdldCBhIHRyYW5zYWN0aW9uIHN0YXJ0ZWQgYW5kIGFsbCB3b3JrIHdpbGwgYmUgZG9uZSBpbnNpZGUgdGhpcyB0cmFuc2FjdGlvbg0KKiBJZiB3ZSBjYWxsIGBtZXRob2RBYCB3ZSB3aWxsIGdldCB0aGUgc2FtZSByZXN1bHQgZm9yIHdoYXRldmVyIGBtZXRob2RBYCBpcyBkb2luZw0KDQpXZWxsLCB3aGlsZSB0aGUgZmlyc3QgYXNzdW1wdGlvbiB3b3JrcyBhcyBzdGF0ZWQgdGhlIHNlY29uZCBpcywgc3VycHJpc2UsIHdyb25nLiBFYWNoIGluZGl2aWR1YWwgY2FsbCB0byB0aGUgbWVudGlvbmVkIEVKQiB3aWxsIHRyaWdnZXIgaXQncyBvd24gdHJhbnNhY3Rpb24uDQogDQpUaGlzIGJlaGF2aW91ciBpcyBkdWUgdG8gdGhlIGZvbGxvd2luZyBub3Qgb2J2aW91cyBhdCBmaXJzdCBnbGFuY2UgZmFjdDogSlNGIHdpbGwgY2FsbCBDREkgbWV0aG9kcyB0aHJvdWdoIGEgcHJveHkgdGhhdCB3cmFwcyB0aGUgYWN0dWFsIGluc3RhbmNlIGFuZCBmaXJlcyBpbnRlcmNlcHRvcnM7IGluIG91ciBzZWNvbmQgY2FzZSBpbnRlcmNlcHRvcnMgaGF2ZSBiZWVuIGFscmVhZHkgZmlyZWQgYmVmb3JlIGBtZXRob2RBYCB3YXMgYWN0dWFsbHkgY2FsbGVkIGFuZCBubyBvdGhlciBpbnRlcmNlcHRvcnMgKHRoYXQgd291bGQgdHJpZ2dlciB0cmFuc2FjdGlvbiBtYW5hZ2VtZW50KSB3aWxsIGJlIGNhbGxlZCBkdWUgdG8gdGhlIGZhY3QgdGhhdCBpbnN0YW5jZSBub3QgcHJveHkgYG1ldGhvZEJgIHdpbGwgYmUgY2FsbGVkLg==

Wednesday, April 13, 2016

SQL Server AlwaysOn

Installing/configuring SQL Server 2014 AlwaysOn Availability Group

The basic desired scenario was a configuration with a primary database (writeable) and a secondary (readable) replica, targeting reports (no actual clustering/fail-over).

The main goal of this post is to point out some prerequisites/main steps not always obvious/present in internet posts/youtube videos.

Installing WSFC (Windows Server fail-over Cluster)

Run the configuration steps as an user with rights to create AD objects. Being a local admin will not suffice. Install first, on both servers (Win 2012 Standard in my case) Failover Cluster feature.

From one of the servers (I run it from main server, I suppose there is no difference) run Create Cluster from Failover Cluster Management.

Add both servers, run validation (for sure you will get some warnings about storage since we do not have any common storage, maybe about the network, as a single point of failure and some warnings about not having the same updates applied on both server; ignore them).

Create the cluster; since in my infrastructure I had DHCP enabled no cluster IP configuration was required.

Configuring AlwaysOn

The operations described below were done with an domain account, member of local Adminsitrators Windows Group and member of sysadmin SQL Server group.

On each server:

  • Open for inbound access TCP port 5022
  • Have the same directory structure where databases file support will be installed
  • Have installed SQL Server 2014, Enterprise or Developer
  • From SQL Server configuration->SQL Server Configuration Manager->SQL Server Service->Properties->AlwaysOn High Availability tab->Enable AlwaysOn Availability Groups
  • Restart SQL Server service

On main server (my choice, can be anywhere in the LAN):

  • create a share \\Server\MyShare with everyone full access (this is required for initial synchronization since we want to leave the default SQL service user assignment (NT Service\MSSQLSERVER)).

Now, on main server open SQL Enterprise Management Studio, go to AlwaysOn High Availability->Availability Groups->New Availability Group Wizard. Choose the database, secondary replica, share for initial synchronization, Endpoint URL, etc. I set (for both servers):

  • Availability Mode to Synchronous (virtual servers are in the same ESX)
  • Failover to Manual
  • Availability to Allow all Connections
  • Readable Secondary Yes

Backup preferences was set to Primary since I already had a full backup daily/log backup every 15 minutes Maintenance Jobs on primary

When creating the group initial join for secondary replica will fail since (as stated above) we will not change the default user accounts for SQL Service.

Go on each server and create/grant rights for pair computer accounts by running:

 /*
 USE master;
 GO
 SELECT name, state_desc, port from sys.tcp_endpoints where type_desc='DATABASE_MIRRORING'
 GO
*/
 USE master;
 GO
 CREATE LOGIN [DOMAIN\SERVER$] FROM WINDOWS;
 GO
 GRANT CONNECT on ENDPOINT::Hadr_endpoint TO [DOMAIN\SERVER$];
 GO

Read more on https://blogs.msdn.microsoft.com/svarukala/2014/03/31/sql-alwayson-failed-to-join-the-database-to-the-availability-group-error-35250/

Go back to SQL Enterprise management studio and manually join the secondary database.

Now we should have a working AlwaysOn Availability Group.

A final challenge now: all database objects are automatically synchronised (DML as well as DDL). What about logins?

We want to add a secondarylogin that has read only access to the database, mainly for accessing it on secondary replica. By adding the login on primary server the AlwaysOn technology will synchronously create the login in secondary database but...no surprise...will not create the SQL Server wide login; so we will get a 'traditional' orphan login. Since the database is read only we can not use the standard fix for orphan users. So, read https://support.microsoft.com/en-us/kb/918992 on how to create clone a SQL Server login from one instance to another (altogether with SID and password hash).

IyBJbnN0YWxsaW5nL2NvbmZpZ3VyaW5nIFNRTCBTZXJ2ZXIgMjAxNCBBbHdheXNPbiBBdmFpbGFiaWxpdHkgR3JvdXANCg0KVGhlIGJhc2ljIGRlc2lyZWQgc2NlbmFyaW8gd2FzIGEgY29uZmlndXJhdGlvbiB3aXRoIGEgcHJpbWFyeSBkYXRhYmFzZSAod3JpdGVhYmxlKSBhbmQgYSBzZWNvbmRhcnkgKHJlYWRhYmxlKSByZXBsaWNhLCB0YXJnZXRpbmcgcmVwb3J0cyAobm8gYWN0dWFsIGNsdXN0ZXJpbmcvZmFpbC1vdmVyKS4gDQoNClRoZSBtYWluIGdvYWwgb2YgdGhpcyBwb3N0IGlzIHRvIHBvaW50IG91dCBzb21lIHByZXJlcXVpc2l0ZXMvbWFpbiBzdGVwcyBub3QgYWx3YXlzIG9idmlvdXMvcHJlc2VudCBpbiBpbnRlcm5ldCBwb3N0cy95b3V0dWJlIHZpZGVvcy4NCg0KIyMgSW5zdGFsbGluZyBXU0ZDIChXaW5kb3dzIFNlcnZlciBmYWlsLW92ZXIgQ2x1c3RlcikNCg0KUnVuIHRoZSBjb25maWd1cmF0aW9uIHN0ZXBzIGFzIGFuIHVzZXIgd2l0aCByaWdodHMgdG8gY3JlYXRlIEFEIG9iamVjdHMuIEJlaW5nIGEgbG9jYWwgYWRtaW4gd2lsbCBub3Qgc3VmZmljZS4NCkluc3RhbGwgZmlyc3QsIG9uIGJvdGggc2VydmVycyAoV2luIDIwMTIgU3RhbmRhcmQgaW4gbXkgY2FzZSkgRmFpbG92ZXIgQ2x1c3RlciBmZWF0dXJlLg0KDQpGcm9tIG9uZSBvZiB0aGUgc2VydmVycyAoSSBydW4gaXQgZnJvbSBtYWluIHNlcnZlciwgSSBzdXBwb3NlIHRoZXJlIGlzIG5vIGRpZmZlcmVuY2UpIHJ1biBDcmVhdGUgQ2x1c3RlciBmcm9tIEZhaWxvdmVyIENsdXN0ZXIgTWFuYWdlbWVudC4NCg0KQWRkIGJvdGggc2VydmVycywgcnVuIHZhbGlkYXRpb24gKGZvciBzdXJlIHlvdSB3aWxsIGdldCBzb21lIHdhcm5pbmdzIGFib3V0IHN0b3JhZ2Ugc2luY2Ugd2UgZG8gbm90IGhhdmUgYW55IGNvbW1vbiBzdG9yYWdlLCBtYXliZSBhYm91dCB0aGUgbmV0d29yaywgYXMgYSBzaW5nbGUgcG9pbnQgb2YgZmFpbHVyZSBhbmQgc29tZSB3YXJuaW5ncyBhYm91dCBub3QgaGF2aW5nIHRoZSBzYW1lIHVwZGF0ZXMgYXBwbGllZCBvbiBib3RoIHNlcnZlcjsgaWdub3JlIHRoZW0pLiANCg0KQ3JlYXRlIHRoZSBjbHVzdGVyOyBzaW5jZSBpbiBteSBpbmZyYXN0cnVjdHVyZSBJIGhhZCBESENQIGVuYWJsZWQgbm8gY2x1c3RlciBJUCBjb25maWd1cmF0aW9uIHdhcyByZXF1aXJlZC4NCg0KIyMgQ29uZmlndXJpbmcgQWx3YXlzT24NCg0KVGhlIG9wZXJhdGlvbnMgZGVzY3JpYmVkIGJlbG93IHdlcmUgZG9uZSB3aXRoIGFuIGRvbWFpbiBhY2NvdW50LCBtZW1iZXIgb2YgbG9jYWwgQWRtaW5zaXRyYXRvcnMgV2luZG93cyBHcm91cCBhbmQgbWVtYmVyIG9mIHN5c2FkbWluIFNRTCBTZXJ2ZXIgZ3JvdXAuDQoNCk9uIGVhY2ggc2VydmVyOg0KLSBPcGVuIGZvciBpbmJvdW5kIGFjY2VzcyBUQ1AgcG9ydCA1MDIyDQotIEhhdmUgdGhlIHNhbWUgZGlyZWN0b3J5IHN0cnVjdHVyZSB3aGVyZSBkYXRhYmFzZXMgZmlsZSBzdXBwb3J0IHdpbGwgYmUgaW5zdGFsbGVkDQotIEhhdmUgaW5zdGFsbGVkIFNRTCBTZXJ2ZXIgMjAxNCwgRW50ZXJwcmlzZSBvciBEZXZlbG9wZXINCi0gRnJvbSBTUUwgU2VydmVyIGNvbmZpZ3VyYXRpb24tPlNRTCBTZXJ2ZXIgQ29uZmlndXJhdGlvbiBNYW5hZ2VyLT5TUUwgU2VydmVyIFNlcnZpY2UtPlByb3BlcnRpZXMtPkFsd2F5c09uIEhpZ2ggQXZhaWxhYmlsaXR5IHRhYi0+RW5hYmxlIEFsd2F5c09uIEF2YWlsYWJpbGl0eSBHcm91cHMNCi0gUmVzdGFydCBTUUwgU2VydmVyIHNlcnZpY2UNCg0KT24gbWFpbiBzZXJ2ZXIgKG15IGNob2ljZSwgY2FuIGJlIGFueXdoZXJlIGluIHRoZSBMQU4pOg0KLSBjcmVhdGUgYSBzaGFyZSBcXFxcU2VydmVyXFxNeVNoYXJlIHdpdGggZXZlcnlvbmUgZnVsbCBhY2Nlc3MgKHRoaXMgaXMgcmVxdWlyZWQgZm9yIGluaXRpYWwgc3luY2hyb25pemF0aW9uIHNpbmNlIHdlIHdhbnQgdG8gbGVhdmUgdGhlIGRlZmF1bHQgU1FMIHNlcnZpY2UgdXNlciBhc3NpZ25tZW50IChOVCBTZXJ2aWNlXE1TU1FMU0VSVkVSKSkuDQoNCk5vdywgb24gbWFpbiBzZXJ2ZXIgb3BlbiBTUUwgRW50ZXJwcmlzZSBNYW5hZ2VtZW50IFN0dWRpbywgZ28gdG8gQWx3YXlzT24gSGlnaCBBdmFpbGFiaWxpdHktPkF2YWlsYWJpbGl0eSBHcm91cHMtPk5ldyBBdmFpbGFiaWxpdHkgR3JvdXAgV2l6YXJkLg0KQ2hvb3NlIHRoZSBkYXRhYmFzZSwgc2Vjb25kYXJ5IHJlcGxpY2EsIHNoYXJlIGZvciBpbml0aWFsIHN5bmNocm9uaXphdGlvbiwgRW5kcG9pbnQgVVJMLCBldGMuIA0KSSBzZXQgKGZvciBib3RoIHNlcnZlcnMpOg0KDQotIEF2YWlsYWJpbGl0eSBNb2RlIHRvIFN5bmNocm9ub3VzICh2aXJ0dWFsIHNlcnZlcnMgYXJlIGluIHRoZSBzYW1lIEVTWCkgDQotIEZhaWxvdmVyIHRvIE1hbnVhbA0KLSBBdmFpbGFiaWxpdHkgdG8gQWxsb3cgYWxsIENvbm5lY3Rpb25zDQotIFJlYWRhYmxlIFNlY29uZGFyeSBZZXMNCg0KQmFja3VwIHByZWZlcmVuY2VzIHdhcyBzZXQgdG8gUHJpbWFyeSBzaW5jZSBJIGFscmVhZHkgaGFkIGEgZnVsbCBiYWNrdXAgZGFpbHkvbG9nIGJhY2t1cCBldmVyeSAxNSBtaW51dGVzIE1haW50ZW5hbmNlIEpvYnMgb24gcHJpbWFyeQ0KDQpXaGVuIGNyZWF0aW5nIHRoZSBncm91cCBpbml0aWFsIGpvaW4gZm9yIHNlY29uZGFyeSByZXBsaWNhIHdpbGwgZmFpbCBzaW5jZSAoYXMgc3RhdGVkIGFib3ZlKSB3ZSB3aWxsIG5vdCBjaGFuZ2UgdGhlIA0KZGVmYXVsdCB1c2VyIGFjY291bnRzIGZvciBTUUwgU2VydmljZS4NCg0KR28gb24gZWFjaCBzZXJ2ZXIgYW5kIGNyZWF0ZS9ncmFudCByaWdodHMgZm9yIHBhaXIgY29tcHV0ZXIgYWNjb3VudHMgYnkgcnVubmluZzoNCg0KYGBgU1FMDQogLyoNCiBVU0UgbWFzdGVyOw0KIEdPDQogU0VMRUNUIG5hbWUsIHN0YXRlX2Rlc2MsIHBvcnQgZnJvbSBzeXMudGNwX2VuZHBvaW50cyB3aGVyZSB0eXBlX2Rlc2M9J0RBVEFCQVNFX01JUlJPUklORycNCiBHTw0KKi8NCiBVU0UgbWFzdGVyOw0KIEdPDQogQ1JFQVRFIExPR0lOIFtET01BSU5cU0VSVkVSJF0gRlJPTSBXSU5ET1dTOw0KIEdPDQogR1JBTlQgQ09OTkVDVCBvbiBFTkRQT0lOVDo6SGFkcl9lbmRwb2ludCBUTyBbRE9NQUlOXFNFUlZFUiRdOw0KIEdPDQpgYGANClJlYWQgbW9yZSBvbiBbaHR0cHM6Ly9ibG9ncy5tc2RuLm1pY3Jvc29mdC5jb20vc3ZhcnVrYWxhLzIwMTQvMDMvMzEvc3FsLWFsd2F5c29uLWZhaWxlZC10by1qb2luLXRoZS1kYXRhYmFzZS10by10aGUtYXZhaWxhYmlsaXR5LWdyb3VwLWVycm9yLTM1MjUwL10oaHR0cHM6Ly9ibG9ncy5tc2RuLm1pY3Jvc29mdC5jb20vc3ZhcnVrYWxhLzIwMTQvMDMvMzEvc3FsLWFsd2F5c29uLWZhaWxlZC10by1qb2luLXRoZS1kYXRhYmFzZS10by10aGUtYXZhaWxhYmlsaXR5LWdyb3VwLWVycm9yLTM1MjUwLykNCg0KR28gYmFjayB0byBTUUwgRW50ZXJwcmlzZSBtYW5hZ2VtZW50IHN0dWRpbyBhbmQgbWFudWFsbHkgam9pbiB0aGUgc2Vjb25kYXJ5IGRhdGFiYXNlLg0KDQpOb3cgd2Ugc2hvdWxkIGhhdmUgYSB3b3JraW5nIEFsd2F5c09uIEF2YWlsYWJpbGl0eSBHcm91cC4NCg0KQSBmaW5hbCBjaGFsbGVuZ2Ugbm93OiBhbGwgZGF0YWJhc2Ugb2JqZWN0cyBhcmUgYXV0b21hdGljYWxseSBzeW5jaHJvbmlzZWQgKERNTCBhcyB3ZWxsIGFzIERETCkuIFdoYXQgYWJvdXQgbG9naW5zPw0KDQpXZSB3YW50IHRvIGFkZCBhIGBzZWNvbmRhcnlsb2dpbmAgdGhhdCBoYXMgcmVhZCBvbmx5IGFjY2VzcyB0byB0aGUgZGF0YWJhc2UsIG1haW5seSBmb3IgYWNjZXNzaW5nIGl0IG9uIHNlY29uZGFyeSByZXBsaWNhLiBCeSBhZGRpbmcgdGhlIGxvZ2luIG9uIHByaW1hcnkgc2VydmVyIHRoZSBBbHdheXNPbiB0ZWNobm9sb2d5IHdpbGwgc3luY2hyb25vdXNseSBjcmVhdGUgdGhlIGxvZ2luIGluIHNlY29uZGFyeSBkYXRhYmFzZSBidXQuLi5ubyBzdXJwcmlzZS4uLndpbGwgbm90IGNyZWF0ZSB0aGUgU1FMIFNlcnZlciB3aWRlIGxvZ2luOyBzbyB3ZSB3aWxsIGdldCBhICd0cmFkaXRpb25hbCcgb3JwaGFuIGxvZ2luLiBTaW5jZSB0aGUgZGF0YWJhc2UgaXMgcmVhZCBvbmx5IHdlIGNhbiBub3QgdXNlIHRoZSBzdGFuZGFyZCBmaXggZm9yIG9ycGhhbiB1c2Vycy4NClNvLCByZWFkIFtodHRwczovL3N1cHBvcnQubWljcm9zb2Z0LmNvbS9lbi11cy9rYi85MTg5OTJdKGh0dHBzOi8vc3VwcG9ydC5taWNyb3NvZnQuY29tL2VuLXVzL2tiLzkxODk5Mikgb24gaG93IHRvIGNyZWF0ZSBjbG9uZSBhIFNRTCBTZXJ2ZXIgbG9naW4gZnJvbSBvbmUgaW5zdGFuY2UgdG8gYW5vdGhlciAoYWx0b2dldGhlciB3aXRoIFNJRCBhbmQgcGFzc3dvcmQgaGFzaCku

Wednesday, March 2, 2016

SonarQube for Dummies

This post applies to SonarQube version 5.3 on Windows

Server Install

  • Download and unzip sonarqube-5.3.zip
  • Go to bin\windows-x86-64 and run StartSonar.bat (I haven't tried running it as service)
  • Browse to http://localhost:9000
  • Go to Administration->System->Update Center->Availabe and install C# (for instance) plug-in
  • Restart

MSBuild.SonarQube.Runner

  • Download and unzip MSBuild.SonarQube.Runner-2.0.zip
  • Add to environment PATH variable MSBuild.SonarQube.Runner-2.0 full path
  • Install MSBuild 14
  • Create a batch file to run MSBuild.SonarQube.Runner.exe, something like:
MSBuild.SonarQube.Runner.exe begin /key:fwk /name:Framework /version:1.0
"C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe" /t:Rebuild main\src\Framework\Framework.sln >1.txt
MSBuild.SonarQube.Runner.exe end
VGhpcyBwb3N0IGFwcGxpZXMgdG8gU29uYXJRdWJlIHZlcnNpb24gNS4zIG9uIFdpbmRvd3MNCg0KIyBTZXJ2ZXIgSW5zdGFsbA0KDQoqIERvd25sb2FkIGFuZCB1bnppcCBgc29uYXJxdWJlLTUuMy56aXBgDQoqIEdvIHRvIGBiaW5cd2luZG93cy14ODYtNjRgIGFuZCBydW4gYFN0YXJ0U29uYXIuYmF0YCAoSSBoYXZlbid0IHRyaWVkIHJ1bm5pbmcgaXQgYXMgc2VydmljZSkNCiogQnJvd3NlIHRvIGh0dHA6Ly9sb2NhbGhvc3Q6OTAwMA0KKiBHbyB0byBgQWRtaW5pc3RyYXRpb24tPlN5c3RlbS0+VXBkYXRlIENlbnRlci0+QXZhaWxhYmVgIGFuZCBpbnN0YWxsIEMjIChmb3IgaW5zdGFuY2UpIHBsdWctaW4NCiogUmVzdGFydA0KDQojIE1TQnVpbGQuU29uYXJRdWJlLlJ1bm5lcg0KDQoqIERvd25sb2FkIGFuZCB1bnppcCBgTVNCdWlsZC5Tb25hclF1YmUuUnVubmVyLTIuMC56aXBgDQoqIEFkZCB0byBlbnZpcm9ubWVudCBQQVRIIHZhcmlhYmxlIGBNU0J1aWxkLlNvbmFyUXViZS5SdW5uZXItMi4wYCBmdWxsIHBhdGgNCiogSW5zdGFsbCBNU0J1aWxkIDE0DQoqIENyZWF0ZSBhIGJhdGNoIGZpbGUgdG8gcnVuIGBNU0J1aWxkLlNvbmFyUXViZS5SdW5uZXIuZXhlYCwgc29tZXRoaW5nIGxpa2U6DQoNCmBgYGNtZA0KTVNCdWlsZC5Tb25hclF1YmUuUnVubmVyLmV4ZSBiZWdpbiAva2V5OmZ3ayAvbmFtZTpGcmFtZXdvcmsgL3ZlcnNpb246MS4wDQoiQzpcUHJvZ3JhbSBGaWxlcyAoeDg2KVxNU0J1aWxkXDE0LjBcQmluXE1TQnVpbGQuZXhlIiAvdDpSZWJ1aWxkIG1haW5cc3JjXEZyYW1ld29ya1xGcmFtZXdvcmsuc2xuID4xLnR4dA0KTVNCdWlsZC5Tb25hclF1YmUuUnVubmVyLmV4ZSBlbmQNCmBgYA0KKiBCcm93c2UgdG8gdGhlIHNwZWNpZmllZCBVUkwgKGxpa2UgaHR0cDovL2xvY2FsaG9zdDo5MDAwL2Rhc2hib2FyZC9pbmRleC9md2spDQoqIEhhcHB5IGZydXN0cmF0aW9uIGFib3V0IGFsbCBraW5kIG9mIHN0dXBpZCAnbWFqb3IvY3JpdGljYWwvYmxvY2tlcicgaXNzdWVzIQ0KDQo=

Tuesday, March 1, 2016

Graylog series, part 1

Graylog Primer

This post applies to Graylog version 1.3.3.

Initial Server Install

This is straightforward: download the virtual appliance, import and start. That's it!

Graylog Collector

This post applies to version 0.4.2.

  • Download graylog-collector-0.4.2.zip archieve and explode it in target directory
  • Rename collector.conf.example to collector.conf; Modify it according to your needs (there are plenty of samples in Graylog documentation)
  • Install/start the service running bin\graylog-collector-service.bat install GraylogCollector and bin\graylog-collector-service.bat start GraylogCollector

Now we have a server side problem: by default there are two pre-installed inputs :appliance-gelf-udp and appliance-syslog-udp while the client Collector supports (for GELF at least) only TCP. So we have to create another input according to our configuration (port mainly).

If everything worked according to the plan we should see now in the Search section our events and every second a nginx sourced event (very annoying).

Server insights

Startup scripts

/opt/graylog/service contains symbolic links to all stratup directories. In /etc/init/graylog-runsvdir.conf there is executed exec /opt/graylog/embedded/bin/runsvdir-start which in turn runs exec env - PATH=$PATH runsvdir -P /opt/graylog/service 'log:.

Every directory pointed by the symbolic links mentioned before contains a run script. For instance in /opt/graylog/sv/graylog-server/ we have

#!/bin/sh
exec 2>&1

umask 077

if [ -f "/opt/graylog/embedded/share/graylog/installation-source.sh" ]; then
    . "/opt/graylog/embedded/share/graylog/installation-source.sh"
fi

export JAVA_HOME=/opt/graylog/embedded/jre
export GRAYLOG_SERVER_JAVA_OPTS="-Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"

# check if mongodb is up
timeout 600 bash -c "until curl -s http://127.0.0.1:27017; do sleep 1; done"
exec chpst -P -U graylog -u graylog /opt/graylog/embedded/bin/authbind $JAVA_HOME/bin/java $GRAYLOG_SERVER_JAVA_OPTS -jar -Dlog4j.configuration=file:///opt/graylog/conf/log4j.xml -Djava.library.path=/opt/graylog/server/lib/sigar/ -Dgraylog2.installation_source=${GRAYLOG_INSTALLATION_SOURCE:=unknown} /opt/graylog/server/graylog.jar server -f /opt/graylog/conf/graylog.conf

Logging

Logging is started through log\run. See svloggelfd documentation on github as well as linux svlogd manual. Notice that UDP is used by svloggelfd.

Actual logs are saved in /var/log/graylog/... according to svlogd rules while svloggelfd sends all data to Graylog UDP input.

Sample log\run content:

#!/bin/sh
exec /opt/graylog/embedded/bin/svloggelfd -H 127.0.0.1:12201 -s graylog-server -e | svlogd -tt /var/log/graylog/server

Disabling nginx Loging forward to Graylog

Just cahnge /opt/graylog/sv/nginx/log/run to:

#!/bin/sh
#exec /opt/graylog/embedded/bin/svloggelfd -H 127.0.0.1:12201 -s nginx -e | svlogd -tt /var/log/graylog/nginx
exec svlogd -tt /var/log/graylog/nginx

and restart Graylog (see below)

Controllig Graylog from command line

graylog-ctl status
graylog-ctl stop
graylog-ctl start

Timezone

Use sudo date to change the default settings for date/time (UTC+0).

Changing timezone :

  • Edit /etc/timezone and change to the desired timezone (like Europe/Bucharest)
  • Run sudo dpkg-reconfigure --frontend noninteractive tzdata

This is not enough; Graylog web interface will still disply events with time in Etc/UTC timezone.

So:

  • Check the startup script for graylog-web (semething like exec chpst -P -U graylog -u graylog /opt/graylog/web/bin/graylog-web-interface -Dconfig.file=/opt/graylog/conf/graylog-web-interface.conf -Dhttp.port=9000 -Dhttp.address=0.0.0.0 -Dpidfile.path=/var/opt/graylog/web.pid -Dlogger.file=/opt/graylog/conf/web-logger.xml
  • Now edit the configuration file (/opt/graylog/conf/graylog-web-interface.conf) and change the timezone property according to your needs
  • This might not be enough: admin user has a fixed UTC timezone configuration so you will have to create a new user and assign the desired timezone or none (so the default just configured before will be used)

Debugging

This is not about bugs but just inspecting the implementation code to clarify things not captured in official documentation

Server

  • Change startup script (/opt/graylog/sv/graylog-server/run) to
#!/bin/sh
exec 2>&1

umask 077

if [ -f "/opt/graylog/embedded/share/graylog/installation-source.sh" ]; then
    . "/opt/graylog/embedded/share/graylog/installation-source.sh"
fi

export JAVA_HOME=/opt/graylog/embedded/jre
# export GRAYLOG_SERVER_JAVA_OPTS="-Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"
export GRAYLOG_SERVER_JAVA_OPTS="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=4242 -Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"

# check if mongodb is up
timeout 600 bash -c "until curl -s http://127.0.0.1:27017; do sleep 1; done"
exec chpst -P -U graylog -u graylog /opt/graylog/embedded/bin/authbind $JAVA_HOME/bin/java $GRAYLOG_SERVER_JAVA_OPTS -jar -Dlog4j.configuration=file:///opt/graylog/conf/log4j.xml -Djava.library.path=/opt/graylog/server/lib/sigar/ -Dgraylog2.installation_source=${GRAYLOG_INSTALLATION_SOURCE:=unknown} /opt/graylog/server/graylog.jar server -f /opt/graylog/conf/graylog.conf
  • Download tag 1.3.3 of graylog-server from github
  • Open pom.xml in your favorite Java IDE
  • Attach to <graylog-server-fqdn>, port 4242
  • Happy inspecting!

Web interface

Debugging web interface is a little bit tricky since it is written in Scala-Play without a maven based project, so:

  • Change the startup script (/opt/graylog/sv/graylog-web/run) to:
#!/bin/sh
exec 2>&1

umask 077
export JAVA_HOME=/opt/graylog/embedded/jre

rm -f /var/opt/graylog/web.pid
exec chpst -P -U graylog -u graylog /opt/graylog/web/bin/graylog-web-interface -jvm-debug 4243 -Dconfig.file=/opt/graylog/conf/graylog-web-interface.conf -Dhttp.port=9000 -Dhttp.address=0.0.0.0 -Dpidfile.path=/var/opt/graylog/web.pid -Dlogger.file=/opt/graylog/conf/web-logger.xml
  • Notice the -jvm-debug 4243 extra parameter
  • Configure your favourite Java IDE to use the source path also from graylog2-web-interface-1.3.3\app (download tag 1.3.3 of deprecated web interface sources)
  • Some functionality refers to classes in graylog2-rest-client module of graylog server project (like org.graylog2.restclient.lib.DateTools containing method getUserTimeZone).
  • Search is performed in renderSearch method of controllers.SearchController class.
  • Render template is graylog2-web-interface-1.3.3\app\views\search\index.scala.html
  • Java script dealing with rendering dates according to the user's timezone is done in app\assets\javascripts\moment-helper.js while gl2UserTimeZone is defined in app\views\partials\navbar.scala.html
IyBHcmF5bG9nIFByaW1lcg0KVGhpcyBwb3N0IGFwcGxpZXMgdG8gR3JheWxvZyB2ZXJzaW9uIDEuMy4zLg0KDQojIyBJbml0aWFsIFNlcnZlciBJbnN0YWxsDQoNClRoaXMgaXMgc3RyYWlnaHRmb3J3YXJkOiBkb3dubG9hZCB0aGUgdmlydHVhbCBhcHBsaWFuY2UsIGltcG9ydCBhbmQgc3RhcnQuIFRoYXQncyBpdCENCg0KIyMgR3JheWxvZyBDb2xsZWN0b3INCg0KVGhpcyBwb3N0IGFwcGxpZXMgdG8gdmVyc2lvbiAwLjQuMi4NCg0KKiBEb3dubG9hZCBncmF5bG9nLWNvbGxlY3Rvci0wLjQuMi56aXAgYXJjaGlldmUgYW5kIGV4cGxvZGUgaXQgaW4gdGFyZ2V0IGRpcmVjdG9yeQ0KKiBSZW5hbWUgYGNvbGxlY3Rvci5jb25mLmV4YW1wbGVgIHRvIGBjb2xsZWN0b3IuY29uZmA7IE1vZGlmeSBpdCBhY2NvcmRpbmcgdG8geW91ciBuZWVkcyAodGhlcmUgYXJlIHBsZW50eSBvZiBzYW1wbGVzIGluIEdyYXlsb2cgZG9jdW1lbnRhdGlvbikNCiogSW5zdGFsbC9zdGFydCB0aGUgc2VydmljZSBydW5uaW5nIGBiaW5cZ3JheWxvZy1jb2xsZWN0b3Itc2VydmljZS5iYXQgaW5zdGFsbCBHcmF5bG9nQ29sbGVjdG9yYCBhbmQgYGJpblxncmF5bG9nLWNvbGxlY3Rvci1zZXJ2aWNlLmJhdCBzdGFydCBHcmF5bG9nQ29sbGVjdG9yYA0KDQpOb3cgd2UgaGF2ZSBhIHNlcnZlciBzaWRlIHByb2JsZW06IGJ5IGRlZmF1bHQgdGhlcmUgYXJlIHR3byBwcmUtaW5zdGFsbGVkIGlucHV0cyA6YGFwcGxpYW5jZS1nZWxmLXVkcGAgYW5kIGBhcHBsaWFuY2Utc3lzbG9nLXVkcGAgd2hpbGUgdGhlIGNsaWVudCBDb2xsZWN0b3Igc3VwcG9ydHMgKGZvciBHRUxGIGF0IGxlYXN0KSBvbmx5IFRDUC4gU28gd2UgaGF2ZSB0byBjcmVhdGUgYW5vdGhlciBpbnB1dCBhY2NvcmRpbmcgdG8gb3VyIGNvbmZpZ3VyYXRpb24gKGBwb3J0YCBtYWlubHkpLg0KDQpJZiBldmVyeXRoaW5nIHdvcmtlZCBhY2NvcmRpbmcgdG8gdGhlIHBsYW4gd2Ugc2hvdWxkIHNlZSBub3cgaW4gdGhlIFNlYXJjaCBzZWN0aW9uIG91ciBldmVudHMgYW5kIGV2ZXJ5IHNlY29uZCBhIGBuZ2lueGAgc291cmNlZCBldmVudCAodmVyeSBhbm5veWluZykuDQoNCiMjIFNlcnZlciBpbnNpZ2h0cw0KDQojIyMgU3RhcnR1cCBzY3JpcHRzDQoNCmAvb3B0L2dyYXlsb2cvc2VydmljZWAgY29udGFpbnMgc3ltYm9saWMgbGlua3MgdG8gYWxsIHN0cmF0dXAgZGlyZWN0b3JpZXMuIEluIGAvZXRjL2luaXQvZ3JheWxvZy1ydW5zdmRpci5jb25mYCB0aGVyZSBpcyBleGVjdXRlZCBgZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3J1bnN2ZGlyLXN0YXJ0YCB3aGljaCBpbiB0dXJuIHJ1bnMgYGV4ZWMgZW52IC0gUEFUSD0kUEFUSCBydW5zdmRpciAtUCAvb3B0L2dyYXlsb2cvc2VydmljZSAnbG9nOmAuDQoNCkV2ZXJ5IGRpcmVjdG9yeSBwb2ludGVkIGJ5IHRoZSBzeW1ib2xpYyBsaW5rcyBtZW50aW9uZWQgYmVmb3JlIGNvbnRhaW5zIGEgYHJ1bmAgc2NyaXB0LiBGb3IgaW5zdGFuY2UgaW4gYC9vcHQvZ3JheWxvZy9zdi9ncmF5bG9nLXNlcnZlci9gIHdlIGhhdmUNCg0KYGBgYmFzaA0KIyEvYmluL3NoDQpleGVjIDI+JjENCg0KdW1hc2sgMDc3DQoNCmlmIFsgLWYgIi9vcHQvZ3JheWxvZy9lbWJlZGRlZC9zaGFyZS9ncmF5bG9nL2luc3RhbGxhdGlvbi1zb3VyY2Uuc2giIF07IHRoZW4NCiAgICAuICIvb3B0L2dyYXlsb2cvZW1iZWRkZWQvc2hhcmUvZ3JheWxvZy9pbnN0YWxsYXRpb24tc291cmNlLnNoIg0KZmkNCg0KZXhwb3J0IEpBVkFfSE9NRT0vb3B0L2dyYXlsb2cvZW1iZWRkZWQvanJlDQpleHBvcnQgR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTPSItWG1zMWcgLVhteDE1MDBtIC1YWDpOZXdSYXRpbz0xIC1YWDpQZXJtU2l6ZT0xMjhtIC1YWDpNYXhQZXJtU2l6ZT0yNTZtIC1zZXJ2ZXIgLVhYOitSZXNpemVUTEFCIC1YWDorVXNlQ29uY01hcmtTd2VlcEdDIC1YWDorQ01TQ29uY3VycmVudE1URW5hYmxlZCAtWFg6K0NNU0NsYXNzVW5sb2FkaW5nRW5hYmxlZCAtWFg6K1VzZVBhck5ld0dDIC1YWDotT21pdFN0YWNrVHJhY2VJbkZhc3RUaHJvdyINCg0KIyBjaGVjayBpZiBtb25nb2RiIGlzIHVwDQp0aW1lb3V0IDYwMCBiYXNoIC1jICJ1bnRpbCBjdXJsIC1zIGh0dHA6Ly8xMjcuMC4wLjE6MjcwMTc7IGRvIHNsZWVwIDE7IGRvbmUiDQpleGVjIGNocHN0IC1QIC1VIGdyYXlsb2cgLXUgZ3JheWxvZyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL2F1dGhiaW5kICRKQVZBX0hPTUUvYmluL2phdmEgJEdSQVlMT0dfU0VSVkVSX0pBVkFfT1BUUyAtamFyIC1EbG9nNGouY29uZmlndXJhdGlvbj1maWxlOi8vL29wdC9ncmF5bG9nL2NvbmYvbG9nNGoueG1sIC1EamF2YS5saWJyYXJ5LnBhdGg9L29wdC9ncmF5bG9nL3NlcnZlci9saWIvc2lnYXIvIC1EZ3JheWxvZzIuaW5zdGFsbGF0aW9uX3NvdXJjZT0ke0dSQVlMT0dfSU5TVEFMTEFUSU9OX1NPVVJDRTo9dW5rbm93bn0gL29wdC9ncmF5bG9nL3NlcnZlci9ncmF5bG9nLmphciBzZXJ2ZXIgLWYgL29wdC9ncmF5bG9nL2NvbmYvZ3JheWxvZy5jb25mDQpgYGANCg0KIyMjIExvZ2dpbmcNCg0KTG9nZ2luZyBpcyBzdGFydGVkIHRocm91Z2ggYGxvZ1xydW5gLiBTZWUgYHN2bG9nZ2VsZmRgIGRvY3VtZW50YXRpb24gb24gZ2l0aHViIGFzIHdlbGwgYXMgbGludXggYHN2bG9nZGAgbWFudWFsLiBOb3RpY2UgdGhhdCBVRFAgaXMgdXNlZCBieSBgc3Zsb2dnZWxmZGAuDQoNCkFjdHVhbCBsb2dzIGFyZSBzYXZlZCBpbiBgL3Zhci9sb2cvZ3JheWxvZy8uLi5gIGFjY29yZGluZyB0byBgc3Zsb2dkYCBydWxlcyB3aGlsZSBgc3Zsb2dnZWxmZGAgc2VuZHMgYWxsIGRhdGEgdG8gR3JheWxvZyBVRFAgaW5wdXQuDQoNClNhbXBsZSBgbG9nXHJ1bmAgY29udGVudDoNCmBgYGJhc2gNCiMhL2Jpbi9zaA0KZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3N2bG9nZ2VsZmQgLUggMTI3LjAuMC4xOjEyMjAxIC1zIGdyYXlsb2ctc2VydmVyIC1lIHwgc3Zsb2dkIC10dCAvdmFyL2xvZy9ncmF5bG9nL3NlcnZlcg0KYGBgDQoNCiMjIyBEaXNhYmxpbmcgbmdpbnggTG9naW5nIGZvcndhcmQgdG8gR3JheWxvZw0KDQpKdXN0IGNhaG5nZSBgL29wdC9ncmF5bG9nL3N2L25naW54L2xvZy9ydW5gIHRvOg0KYGBgYmFzaA0KIyEvYmluL3NoDQojZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3N2bG9nZ2VsZmQgLUggMTI3LjAuMC4xOjEyMjAxIC1zIG5naW54IC1lIHwgc3Zsb2dkIC10dCAvdmFyL2xvZy9ncmF5bG9nL25naW54DQpleGVjIHN2bG9nZCAtdHQgL3Zhci9sb2cvZ3JheWxvZy9uZ2lueA0KYGBgDQphbmQgcmVzdGFydCBHcmF5bG9nIChzZWUgYmVsb3cpDQoNCiMjIyBDb250cm9sbGlnIEdyYXlsb2cgZnJvbSBjb21tYW5kIGxpbmUNCg0KYGBgYmFzaA0KZ3JheWxvZy1jdGwgc3RhdHVzDQpncmF5bG9nLWN0bCBzdG9wDQpncmF5bG9nLWN0bCBzdGFydA0KYGBgDQoNCiMjIyBUaW1lem9uZQ0KDQpVc2UgYHN1ZG8gZGF0ZWAgdG8gY2hhbmdlIHRoZSBkZWZhdWx0IHNldHRpbmdzIGZvciBkYXRlL3RpbWUgKFVUQyswKS4NCg0KQ2hhbmdpbmcgdGltZXpvbmUgOg0KKiBFZGl0IC9ldGMvdGltZXpvbmUgYW5kIGNoYW5nZSB0byB0aGUgZGVzaXJlZCB0aW1lem9uZSAobGlrZSBFdXJvcGUvQnVjaGFyZXN0KQ0KKiBSdW4gc3VkbyBkcGtnLXJlY29uZmlndXJlIC0tZnJvbnRlbmQgbm9uaW50ZXJhY3RpdmUgdHpkYXRhDQoNClRoaXMgaXMgbm90IGVub3VnaDsgR3JheWxvZyB3ZWIgaW50ZXJmYWNlIHdpbGwgc3RpbGwgZGlzcGx5IGV2ZW50cyB3aXRoIHRpbWUgaW4gRXRjL1VUQyB0aW1lem9uZS4NCg0KU286DQoqIENoZWNrIHRoZSBzdGFydHVwIHNjcmlwdCBmb3IgZ3JheWxvZy13ZWIgKHNlbWV0aGluZyBsaWtlIGBleGVjIGNocHN0IC1QIC1VIGdyYXlsb2cgLXUgZ3JheWxvZyAvb3B0L2dyYXlsb2cvd2ViL2Jpbi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UgLURjb25maWcuZmlsZT0vb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UuY29uZiAtRGh0dHAucG9ydD05MDAwIC1EaHR0cC5hZGRyZXNzPTAuMC4wLjAgLURwaWRmaWxlLnBhdGg9L3Zhci9vcHQvZ3JheWxvZy93ZWIucGlkIC1EbG9nZ2VyLmZpbGU9L29wdC9ncmF5bG9nL2NvbmYvd2ViLWxvZ2dlci54bWxgDQoqIE5vdyBlZGl0IHRoZSBjb25maWd1cmF0aW9uIGZpbGUgKGAvb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UuY29uZmApIGFuZCBjaGFuZ2UgdGhlIHRpbWV6b25lIHByb3BlcnR5IGFjY29yZGluZyB0byB5b3VyIG5lZWRzDQoqIFRoaXMgbWlnaHQgbm90IGJlIGVub3VnaDogYWRtaW4gdXNlciBoYXMgYSBmaXhlZCBVVEMgdGltZXpvbmUgY29uZmlndXJhdGlvbiBzbyB5b3Ugd2lsbCBoYXZlIHRvIGNyZWF0ZSBhIG5ldyB1c2VyIGFuZCBhc3NpZ24gdGhlIGRlc2lyZWQgdGltZXpvbmUgb3Igbm9uZSAoc28gdGhlIGRlZmF1bHQganVzdCBjb25maWd1cmVkIGJlZm9yZSB3aWxsIGJlIHVzZWQpDQoNCiMjIyBEZWJ1Z2dpbmcNCg0KVGhpcyBpcyBub3QgYWJvdXQgYnVncyBidXQganVzdCBpbnNwZWN0aW5nIHRoZSBpbXBsZW1lbnRhdGlvbiBjb2RlIHRvIGNsYXJpZnkgdGhpbmdzIG5vdCBjYXB0dXJlZCBpbiBvZmZpY2lhbCBkb2N1bWVudGF0aW9uDQoNCiMjIyMgU2VydmVyDQoNCiogQ2hhbmdlIHN0YXJ0dXAgc2NyaXB0IChgL29wdC9ncmF5bG9nL3N2L2dyYXlsb2ctc2VydmVyL3J1bmApIHRvDQpgYGBiYXNoDQojIS9iaW4vc2gNCmV4ZWMgMj4mMQ0KDQp1bWFzayAwNzcNCg0KaWYgWyAtZiAiL29wdC9ncmF5bG9nL2VtYmVkZGVkL3NoYXJlL2dyYXlsb2cvaW5zdGFsbGF0aW9uLXNvdXJjZS5zaCIgXTsgdGhlbg0KICAgIC4gIi9vcHQvZ3JheWxvZy9lbWJlZGRlZC9zaGFyZS9ncmF5bG9nL2luc3RhbGxhdGlvbi1zb3VyY2Uuc2giDQpmaQ0KDQpleHBvcnQgSkFWQV9IT01FPS9vcHQvZ3JheWxvZy9lbWJlZGRlZC9qcmUNCiMgZXhwb3J0IEdSQVlMT0dfU0VSVkVSX0pBVkFfT1BUUz0iLVhtczFnIC1YbXgxNTAwbSAtWFg6TmV3UmF0aW89MSAtWFg6UGVybVNpemU9MTI4bSAtWFg6TWF4UGVybVNpemU9MjU2bSAtc2VydmVyIC1YWDorUmVzaXplVExBQiAtWFg6K1VzZUNvbmNNYXJrU3dlZXBHQyAtWFg6K0NNU0NvbmN1cnJlbnRNVEVuYWJsZWQgLVhYOitDTVNDbGFzc1VubG9hZGluZ0VuYWJsZWQgLVhYOitVc2VQYXJOZXdHQyAtWFg6LU9taXRTdGFja1RyYWNlSW5GYXN0VGhyb3ciDQpleHBvcnQgR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTPSItWGRlYnVnIC1Ybm9hZ2VudCAtWHJ1bmpkd3A6dHJhbnNwb3J0PWR0X3NvY2tldCxzZXJ2ZXI9eSxzdXNwZW5kPW4sYWRkcmVzcz00MjQyIC1YbXMxZyAtWG14MTUwMG0gLVhYOk5ld1JhdGlvPTEgLVhYOlBlcm1TaXplPTEyOG0gLVhYOk1heFBlcm1TaXplPTI1Nm0gLXNlcnZlciAtWFg6K1Jlc2l6ZVRMQUIgLVhYOitVc2VDb25jTWFya1N3ZWVwR0MgLVhYOitDTVNDb25jdXJyZW50TVRFbmFibGVkIC1YWDorQ01TQ2xhc3NVbmxvYWRpbmdFbmFibGVkIC1YWDorVXNlUGFyTmV3R0MgLVhYOi1PbWl0U3RhY2tUcmFjZUluRmFzdFRocm93Ig0KDQojIGNoZWNrIGlmIG1vbmdvZGIgaXMgdXANCnRpbWVvdXQgNjAwIGJhc2ggLWMgInVudGlsIGN1cmwgLXMgaHR0cDovLzEyNy4wLjAuMToyNzAxNzsgZG8gc2xlZXAgMTsgZG9uZSINCmV4ZWMgY2hwc3QgLVAgLVUgZ3JheWxvZyAtdSBncmF5bG9nIC9vcHQvZ3JheWxvZy9lbWJlZGRlZC9iaW4vYXV0aGJpbmQgJEpBVkFfSE9NRS9iaW4vamF2YSAkR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTIC1qYXIgLURsb2c0ai5jb25maWd1cmF0aW9uPWZpbGU6Ly8vb3B0L2dyYXlsb2cvY29uZi9sb2c0ai54bWwgLURqYXZhLmxpYnJhcnkucGF0aD0vb3B0L2dyYXlsb2cvc2VydmVyL2xpYi9zaWdhci8gLURncmF5bG9nMi5pbnN0YWxsYXRpb25fc291cmNlPSR7R1JBWUxPR19JTlNUQUxMQVRJT05fU09VUkNFOj11bmtub3dufSAvb3B0L2dyYXlsb2cvc2VydmVyL2dyYXlsb2cuamFyIHNlcnZlciAtZiAvb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLmNvbmYNCmBgYA0KKiBEb3dubG9hZCB0YWcgMS4zLjMgb2YgZ3JheWxvZy1zZXJ2ZXIgZnJvbSBnaXRodWINCiogT3BlbiBwb20ueG1sIGluIHlvdXIgZmF2b3JpdGUgSmF2YSBJREUNCiogQXR0YWNoIHRvIDxncmF5bG9nLXNlcnZlci1mcWRuPiwgcG9ydCA0MjQyDQoqIEhhcHB5IGluc3BlY3RpbmchDQoNCiMjIyMgV2ViIGludGVyZmFjZQ0KDQpEZWJ1Z2dpbmcgd2ViIGludGVyZmFjZSBpcyBhIGxpdHRsZSBiaXQgdHJpY2t5IHNpbmNlIGl0IGlzIHdyaXR0ZW4gaW4gU2NhbGEtUGxheSB3aXRob3V0IGEgbWF2ZW4gYmFzZWQgcHJvamVjdCwgc286DQoNCiogQ2hhbmdlIHRoZSBzdGFydHVwIHNjcmlwdCAoYC9vcHQvZ3JheWxvZy9zdi9ncmF5bG9nLXdlYi9ydW5gKSB0bzoNCmBgYGJhc2gNCiMhL2Jpbi9zaA0KZXhlYyAyPiYxDQoNCnVtYXNrIDA3Nw0KZXhwb3J0IEpBVkFfSE9NRT0vb3B0L2dyYXlsb2cvZW1iZWRkZWQvanJlDQoNCnJtIC1mIC92YXIvb3B0L2dyYXlsb2cvd2ViLnBpZA0KZXhlYyBjaHBzdCAtUCAtVSBncmF5bG9nIC11IGdyYXlsb2cgL29wdC9ncmF5bG9nL3dlYi9iaW4vZ3JheWxvZy13ZWItaW50ZXJmYWNlIC1qdm0tZGVidWcgNDI0MyAtRGNvbmZpZy5maWxlPS9vcHQvZ3JheWxvZy9jb25mL2dyYXlsb2ctd2ViLWludGVyZmFjZS5jb25mIC1EaHR0cC5wb3J0PTkwMDAgLURodHRwLmFkZHJlc3M9MC4wLjAuMCAtRHBpZGZpbGUucGF0aD0vdmFyL29wdC9ncmF5bG9nL3dlYi5waWQgLURsb2dnZXIuZmlsZT0vb3B0L2dyYXlsb2cvY29uZi93ZWItbG9nZ2VyLnhtbA0KYGBgDQoqIE5vdGljZSB0aGUgYC1qdm0tZGVidWcgNDI0M2AgZXh0cmEgcGFyYW1ldGVyDQoqIENvbmZpZ3VyZSB5b3VyIGZhdm91cml0ZSBKYXZhIElERSB0byB1c2UgdGhlIHNvdXJjZSBwYXRoIGFsc28gZnJvbSBgZ3JheWxvZzItd2ViLWludGVyZmFjZS0xLjMuM1xhcHBgIChkb3dubG9hZCB0YWcgMS4zLjMgb2YgZGVwcmVjYXRlZCB3ZWIgaW50ZXJmYWNlIHNvdXJjZXMpDQoqIFNvbWUgZnVuY3Rpb25hbGl0eSByZWZlcnMgdG8gY2xhc3NlcyBpbiBgZ3JheWxvZzItcmVzdC1jbGllbnRgIG1vZHVsZSBvZiBncmF5bG9nIHNlcnZlciBwcm9qZWN0ICAobGlrZSBgb3JnLmdyYXlsb2cyLnJlc3RjbGllbnQubGliLkRhdGVUb29sc2AgY29udGFpbmluZyBtZXRob2QgYGdldFVzZXJUaW1lWm9uZWApLg0KKiBTZWFyY2ggaXMgcGVyZm9ybWVkIGluIGByZW5kZXJTZWFyY2hgIG1ldGhvZCBvZiBgY29udHJvbGxlcnMuU2VhcmNoQ29udHJvbGxlcmAgY2xhc3MuDQoqIFJlbmRlciB0ZW1wbGF0ZSBpcyBgZ3JheWxvZzItd2ViLWludGVyZmFjZS0xLjMuM1xhcHBcdmlld3Ncc2VhcmNoXGluZGV4LnNjYWxhLmh0bWxgDQoqIEphdmEgc2NyaXB0IGRlYWxpbmcgd2l0aCByZW5kZXJpbmcgZGF0ZXMgYWNjb3JkaW5nIHRvIHRoZSB1c2VyJ3MgdGltZXpvbmUgaXMgZG9uZSBpbiBgYXBwXGFzc2V0c1xqYXZhc2NyaXB0c1xtb21lbnQtaGVscGVyLmpzYCB3aGlsZSBgZ2wyVXNlclRpbWVab25lYCBpcyBkZWZpbmVkIGluIGBhcHBcdmlld3NccGFydGlhbHNcbmF2YmFyLnNjYWxhLmh0bWxg

Tuesday, November 10, 2015

Shibboleth-We Are Not Done Yet-Part 3

This will be quick: by applying the third solution described in the previous post the SP worked with no problem. The solution was based on ISA Server, Kentor.AuthServices and ASP.NET MVC. The ability to run sites on default HTTPS port made things simple. Linux rulez!

BUt...we are not done yet! Logout waits to be implemented.

VGhpcyB3aWxsIGJlIHF1aWNrOiBieSBhcHBseWluZyB0aGUgdGhpcmQgc29sdXRpb24gZGVzY3JpYmVkIGluIHRoZSBwcmV2aW91cyBwb3N0IHRoZSBTUCB3b3JrZWQgd2l0aCBubyBwcm9ibGVtLiBUaGUgc29sdXRpb24gd2FzIGJhc2VkIG9uIElTQSBTZXJ2ZXIsIEtlbnRvci5BdXRoU2VydmljZXMgYW5kIEFTUC5ORVQgTVZDLiBUaGUgYWJpbGl0eSB0byBydW4gc2l0ZXMgb24gZGVmYXVsdCBIVFRQUyBwb3J0IG1hZGUgdGhpbmdzIHNpbXBsZS4gTGludXggcnVsZXohDQoNCkJVdC4uLndlIGFyZSBub3QgZG9uZSB5ZXQhIExvZ291dCB3YWl0cyB0byBiZSBpbXBsZW1lbnRlZC4=

Sunday, November 8, 2015

Shibboleth-We Are Not Done Yet-Part 2

First tests were done in a LAN type environment; unfortunately life is not so simple: targeted solution should support users outside organization as well as Service Providers hosted in third party data centres. Since we do not want our IdP to be published directly on the Internet we have to use a reverse proxy which breaks our working solution. This might be particular (in terms of lack of out of the box solution-it might exist but I was unable to figure one) to ISA server but it looks (after google-ing a little bit) that a lot of people faced the same problem.

The Problem

Let's suppose our proxy has the public FQDN of proxy.mycompany.com while the internal FQDN of our Shibboleth IdP is idp.mycompany.local.

As a consequence external SP will have IdP metadata configured with URLs like https://proxy.mycompany.com/ipd/profile/SAML2/Redirect/SSO. When a request comes, the SAMLRequest will have AuthnRequest tag the attribute Destination set to this address while the proxy will forward (under default configuration) in relayed request's Host header the value of idp.mycompany.local:8443. This behaviour will lead to the failure of the request with this error message logged (sort of...) "SAML message intended destination endpoint 'https://proxy.mycompany.com/ipd/profile/SAML2/Redirect/SSO' did not match the recipient endpoint '{https://idp.mycompany.local:8443/ipd/profile/SAML2/Redirect/SSO'".

The error is generated in class org.opensaml.saml.common.binding.security.impl.ReceivedEndpointSecurityHandler, method checkEndpointURI.

We have two issues that make finding a solution non trivial:

  • We want to use HTTPS also for accessing our internal IdP (dropping this requirement will not actually lead to a working solution due to second issue)
  • Port translation from default 443 (not specified in external URL) to 8443

Usage of HTTPS for internal IdP

Initially Jetty was configured to use for SSL a certificate with common name of idp.mycompany.local with matches the URL configured in ISA publishing rule.

First solution to try was to change the default ISA behaviour in terms of Host header: check the option to forward the original header but another problem popped up: Jetty was refusing the connection since the Host header did not match any valid certificates in terms of common name.

Second solution tried was to generate a trusted (internally) certificate with common name of proxy.mycompany.com. Now ISA would reject it because it attempted a SSL connection on idp.mycompany.local and received a certificate on proxy.mycompany.com.

Third solution (I haven't tried because it would not solve the problem anyway due to port translation) would be:

  • The certificate installed on ISA should be wild card type
  • We have control on both external and internal DNS
  • Add an external DNS alias of idp.mycompany.com resolving to the ip of the external interface of ISA
  • Add an internal DNS alias of idp.mycompany.com resolving to the ip of the internal interface of ISA
  • Generate a trusted certificate for idp.mycompany.com and configure Jetty to use it
  • Add a publish rule for idp.mycompany.com that forwards requests to the IP of idp.mycompany.local and has filled in "This rule appliest to this published site:" idp.mycompany.com so that header Host will contain this value.

Now we should have circumvented our first problem but it will be of no help since the checkEndpointURI (through calling compareEndpointURIs, etc) will do a string comparison which will obviously fail due to the additional substring ":8443" in the actual receiverEndpoint.

8443 Port

Looking at Jetty documentation we quickly find four solutions to overcome this problem:

  • Start Jetty as root and change HTPPS port to 443; we do not want to do that (would we?)
  • Solutions based on ipchains/iptable which is not actually what we need: there will be an internal kernel routing from 443 to 8443 and at the end of the day we will get into the same problem as mentioned earlier
  • Configuring SetUID Jetty feature; this looks like solving our problem but needs some additional know how which I was not willing to acquire (native compiling, running the service as root...)

The Solution

The solution was to use Jetty rewrite handler capabilities (http://www.eclipse.org/jetty/documentation/current/rewrite-handler.html).

Unfortunately, none of the out of the box handlers provided the required functionality so I quickly wrote one.

Before going into into writing/deploying the custom handler we have to perform some prerequisites:

  • Check to see if org.eclipse.jetty.server.Request supports updating hots and port properties (since standard HttpServletRequest does not)
  • Check how to specify updated port so that it will not be included in receiverEndpoint string
  • Check how rewrite handler is activated

First create a maven project having jetty-rewrite dependency and download sources...

        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-rewrite</artifactId>
            <version>9.3.5.v20151012</version>
        </dependency>

First item check passed since looks like we could use

        Request jettyRequest=(Request) request;
        jettyRequest.getMetaData().getURI().setAuthority(host, port);

Second item check was also quick: org.eclipse.jetty.util.appendSchemeHostPort shows that if specifying default ports (depending on schema) or 0 they will not be included.

Last item requires the following steps:

Add

--module=rewrite
etc/rewrite-rules.xml

in $JETTY_BASE/start.ini, where rewrite-rules.xml contains the actual rules and should be located in $JETTY_BASE/etc/rewrite-rules.xml:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">

<!-- =============================================================== -->
<!-- Configure the demos                                             -->
<!-- =============================================================== -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">

  <!-- ============================================================= -->
  <!-- Add rewrite rules                                             -->
  <!-- ============================================================= -->
  <Ref refid="Rewrite">

      <!-- protect favicon handling -->
      <Call name="addRule">
        <Arg>
          <New class="ro.totalsoft.jetty.hostheaderrewriter.HostHeaderRewriter">
            <Set name="header">Host</Set>
            <Set name="host">proxy.mycompany.com</Set>
            <Set name="port">0</Set>
            <Set name="terminating">true</Set>
          </New>
        </Arg>
      </Call>
  </Ref>
</Configure>

In order to be selective (if we host also other applications in the same Jetty instance) we should also add the filtering condition

<Set name="headerValue">idp.mycompany.local:8443</Set>

The actual rewrite handler definition is located in $JETTY_HOME/etc/jetty-rewrite.xml as can be seen by inspection the $JETTY_HOME/module/rewrite.mod module definition file.

Remark: looking (as a newbie) in jetty-rewrite.xml might raise a question: what is oldhandler about? The naming is misleading, it should be something like firsthandler since it refers to the first handler in the chain of Jetty handlers. After Jetty applies the rewrite handler definition this will become the first handler in chain.

Now (supposing we have written our own header rule) we have to deploy it in the classpath known by Jetty. To do this we have to add the ext module:

--module=ext

--module=ext
The ext module will enable the lib/ext/*.jar logic.

If this module is activated, then all jar files found in the lib/ext/ paths will be automatically added to the Jetty Server Classpath.

Afterwards we can copy our jar to $JETTY_BASE/lib/ext/ and restart Jetty

**Important remark **: due to a bug (omission) in the logic that activates rules inside the rewrite handler if there is no rule defined a NullPointerException will be thrown in class org.eclipse.jetty.rewrite.handler.RuleContainer, method apply:

    protected String apply(String target, HttpServletRequest request, HttpServletResponse response) throws IOException
    {
        boolean original_set=_originalPathAttribute==null;
                
        for (Rule rule : _rules)
        {
            String applied=rule.matchAndApply(target,request, response);
            if (applied!=null)
            {       

If no rules are defined we will land here with _rules being null...

Epilogue

Ww are not done yet but have made significant progress having IdP side working. As we can assume, we will face same problem on SP side (we do not really want web sites exposed directly on the Internet).

So, next post about Kentor SSO. I would expect to apply the solution with external/internal DNS and using 443 default port (Windows/IIS looks more permissive in this area)

The Code

package com.mycompany.jetty.hostheaderrewriter;

import java.io.IOException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.eclipse.jetty.rewrite.handler.HeaderRule;
import org.eclipse.jetty.server.Request;

public class HostHeaderRewriter extends HeaderRule{
    String host;
    int port=0;

    public String getHost() {
        return host;
    }
    public void setHost(String host) {
        this.host = host;
    }

    public int getPort() {
        return port;
    }
    public void setPort(int port) {
        this.port = port;
    }
    
    @Override
    protected String apply(String target, String value, HttpServletRequest request, HttpServletResponse response) throws IOException {
        Request jettyRequest=(Request) request;
        jettyRequest.getMetaData().getURI().setAuthority(host, port);
        return target;
    }
}
Rmlyc3QgdGVzdHMgd2VyZSBkb25lIGluIGEgTEFOIHR5cGUgZW52aXJvbm1lbnQ7IHVuZm9ydHVuYXRlbHkgbGlmZSBpcyBub3Qgc28gc2ltcGxlOiB0YXJnZXRlZCBzb2x1dGlvbiBzaG91bGQgc3VwcG9ydCB1c2VycyBvdXRzaWRlIG9yZ2FuaXphdGlvbiBhcyB3ZWxsIGFzIFNlcnZpY2UgUHJvdmlkZXJzIGhvc3RlZCBpbiB0aGlyZCBwYXJ0eSBkYXRhIGNlbnRyZXMuIFNpbmNlIHdlIGRvIG5vdCB3YW50IG91ciBJZFAgdG8gYmUgcHVibGlzaGVkIGRpcmVjdGx5IG9uIHRoZSBJbnRlcm5ldCB3ZSBoYXZlIHRvIHVzZSBhIHJldmVyc2UgcHJveHkgd2hpY2ggYnJlYWtzIG91ciB3b3JraW5nIHNvbHV0aW9uLiBUaGlzIG1pZ2h0IGJlIHBhcnRpY3VsYXIgKGluIHRlcm1zIG9mIGxhY2sgb2Ygb3V0IG9mIHRoZSBib3ggc29sdXRpb24taXQgbWlnaHQgZXhpc3QgYnV0IEkgd2FzIHVuYWJsZSB0byBmaWd1cmUgb25lKSB0byBJU0Egc2VydmVyIGJ1dCBpdCBsb29rcyAoYWZ0ZXIgZ29vZ2xlLWluZyBhIGxpdHRsZSBiaXQpIHRoYXQgYSBsb3Qgb2YgcGVvcGxlIGZhY2VkIHRoZSBzYW1lIHByb2JsZW0uDQoNCiMgVGhlIFByb2JsZW0NCg0KTGV0J3Mgc3VwcG9zZSBvdXIgcHJveHkgaGFzIHRoZSBwdWJsaWMgRlFETiBvZiAqKnByb3h5Lm15Y29tcGFueS5jb20qKiB3aGlsZSB0aGUgaW50ZXJuYWwgRlFETiBvZiBvdXIgU2hpYmJvbGV0aCBJZFAgaXMgKippZHAubXljb21wYW55LmxvY2FsKiouIA0KDQpBcyBhIGNvbnNlcXVlbmNlIGV4dGVybmFsIFNQIHdpbGwgaGF2ZSBJZFAgbWV0YWRhdGEgY29uZmlndXJlZCB3aXRoIFVSTHMgbGlrZSBgaHR0cHM6Ly9wcm94eS5teWNvbXBhbnkuY29tL2lwZC9wcm9maWxlL1NBTUwyL1JlZGlyZWN0L1NTT2AuIFdoZW4gYSByZXF1ZXN0IGNvbWVzLCB0aGUgU0FNTFJlcXVlc3Qgd2lsbCBoYXZlIEF1dGhuUmVxdWVzdCB0YWcgdGhlIGF0dHJpYnV0ZSAqKkRlc3RpbmF0aW9uKiogc2V0IHRvIHRoaXMgYWRkcmVzcyB3aGlsZSB0aGUgcHJveHkgd2lsbCBmb3J3YXJkICh1bmRlciBkZWZhdWx0IGNvbmZpZ3VyYXRpb24pIGluIHJlbGF5ZWQgcmVxdWVzdCdzICoqSG9zdCoqIGhlYWRlciB0aGUgdmFsdWUgb2YgKippZHAubXljb21wYW55LmxvY2FsOjg0NDMqKi4gDQpUaGlzIGJlaGF2aW91ciB3aWxsIGxlYWQgdG8gdGhlIGZhaWx1cmUgb2YgdGhlIHJlcXVlc3Qgd2l0aCB0aGlzIGVycm9yIG1lc3NhZ2UgbG9nZ2VkIChzb3J0IG9mLi4uKSAqIlNBTUwgbWVzc2FnZSBpbnRlbmRlZCBkZXN0aW5hdGlvbiBlbmRwb2ludCAnaHR0cHM6Ly9wcm94eS5teWNvbXBhbnkuY29tL2lwZC9wcm9maWxlL1NBTUwyL1JlZGlyZWN0L1NTTycgZGlkIG5vdCBtYXRjaCB0aGUgcmVjaXBpZW50IGVuZHBvaW50ICd7aHR0cHM6Ly9pZHAubXljb21wYW55LmxvY2FsOjg0NDMvaXBkL3Byb2ZpbGUvU0FNTDIvUmVkaXJlY3QvU1NPJyIqLg0KDQpUaGUgZXJyb3IgaXMgZ2VuZXJhdGVkIGluIGNsYXNzIGBvcmcub3BlbnNhbWwuc2FtbC5jb21tb24uYmluZGluZy5zZWN1cml0eS5pbXBsLlJlY2VpdmVkRW5kcG9pbnRTZWN1cml0eUhhbmRsZXJgLCBtZXRob2QgYGNoZWNrRW5kcG9pbnRVUklgLg0KDQpXZSBoYXZlIHR3byBpc3N1ZXMgdGhhdCBtYWtlIGZpbmRpbmcgYSBzb2x1dGlvbiBub24gdHJpdmlhbDoNCg0KKiBXZSB3YW50IHRvIHVzZSBIVFRQUyBhbHNvIGZvciBhY2Nlc3Npbmcgb3VyIGludGVybmFsIElkUCAoZHJvcHBpbmcgdGhpcyByZXF1aXJlbWVudCB3aWxsIG5vdCBhY3R1YWxseSBsZWFkIHRvIGEgd29ya2luZyBzb2x1dGlvbiBkdWUgdG8gc2Vjb25kIGlzc3VlKQ0KKiBQb3J0IHRyYW5zbGF0aW9uIGZyb20gZGVmYXVsdCA0NDMgKG5vdCBzcGVjaWZpZWQgaW4gZXh0ZXJuYWwgVVJMKSB0byA4NDQzDQoNCiMjIFVzYWdlIG9mIEhUVFBTIGZvciBpbnRlcm5hbCBJZFANCg0KSW5pdGlhbGx5IEpldHR5IHdhcyBjb25maWd1cmVkIHRvIHVzZSBmb3IgU1NMIGEgY2VydGlmaWNhdGUgd2l0aCBjb21tb24gbmFtZSBvZiAqKmlkcC5teWNvbXBhbnkubG9jYWwqKiB3aXRoIG1hdGNoZXMgdGhlIFVSTCBjb25maWd1cmVkIGluIElTQSBwdWJsaXNoaW5nIHJ1bGUuDQoNCkZpcnN0IHNvbHV0aW9uIHRvIHRyeSB3YXMgdG8gY2hhbmdlIHRoZSBkZWZhdWx0IElTQSBiZWhhdmlvdXIgaW4gdGVybXMgb2YgKipIb3N0KiogaGVhZGVyOiBjaGVjayB0aGUgb3B0aW9uIHRvIGZvcndhcmQgdGhlIG9yaWdpbmFsIGhlYWRlciBidXQgYW5vdGhlciBwcm9ibGVtIHBvcHBlZCB1cDogSmV0dHkgd2FzIHJlZnVzaW5nIHRoZSBjb25uZWN0aW9uIHNpbmNlIHRoZSAqKkhvc3QqKiBoZWFkZXIgZGlkIG5vdCBtYXRjaCBhbnkgdmFsaWQgY2VydGlmaWNhdGVzIGluIHRlcm1zIG9mIGNvbW1vbiBuYW1lLg0KDQpTZWNvbmQgc29sdXRpb24gdHJpZWQgd2FzIHRvIGdlbmVyYXRlIGEgdHJ1c3RlZCAoaW50ZXJuYWxseSkgY2VydGlmaWNhdGUgd2l0aCBjb21tb24gbmFtZSBvZiAqKnByb3h5Lm15Y29tcGFueS5jb20qKi4gTm93IElTQSB3b3VsZCByZWplY3QgaXQgYmVjYXVzZSBpdCBhdHRlbXB0ZWQgYSBTU0wgY29ubmVjdGlvbiBvbiAqKmlkcC5teWNvbXBhbnkubG9jYWwqKiBhbmQgcmVjZWl2ZWQgYSBjZXJ0aWZpY2F0ZSBvbiAqKnByb3h5Lm15Y29tcGFueS5jb20qKi4NCg0KVGhpcmQgc29sdXRpb24gKEkgaGF2ZW4ndCB0cmllZCBiZWNhdXNlIGl0IHdvdWxkIG5vdCBzb2x2ZSB0aGUgcHJvYmxlbSBhbnl3YXkgZHVlIHRvIHBvcnQgdHJhbnNsYXRpb24pIHdvdWxkIGJlOg0KDQoqIFRoZSBjZXJ0aWZpY2F0ZSBpbnN0YWxsZWQgb24gSVNBIHNob3VsZCBiZSB3aWxkIGNhcmQgdHlwZQ0KKiBXZSBoYXZlIGNvbnRyb2wgb24gYm90aCBleHRlcm5hbCBhbmQgaW50ZXJuYWwgRE5TDQoqIEFkZCBhbiBleHRlcm5hbCBETlMgYWxpYXMgb2YgKippZHAubXljb21wYW55LmNvbSoqIHJlc29sdmluZyB0byB0aGUgaXAgb2YgdGhlIGV4dGVybmFsIGludGVyZmFjZSBvZiBJU0ENCiogQWRkIGFuIGludGVybmFsIEROUyBhbGlhcyBvZiAqKmlkcC5teWNvbXBhbnkuY29tKiogcmVzb2x2aW5nIHRvIHRoZSBpcCBvZiB0aGUgaW50ZXJuYWwgaW50ZXJmYWNlIG9mIElTQQ0KKiBHZW5lcmF0ZSBhIHRydXN0ZWQgY2VydGlmaWNhdGUgZm9yICoqaWRwLm15Y29tcGFueS5jb20qKiBhbmQgY29uZmlndXJlIEpldHR5IHRvIHVzZSBpdA0KKiBBZGQgYSBwdWJsaXNoIHJ1bGUgZm9yICoqaWRwLm15Y29tcGFueS5jb20qKiB0aGF0IGZvcndhcmRzIHJlcXVlc3RzIHRvIHRoZSBJUCBvZiAgKippZHAubXljb21wYW55LmxvY2FsKiogYW5kIGhhcyBmaWxsZWQgaW4gIlRoaXMgcnVsZSBhcHBsaWVzdCB0byB0aGlzIHB1Ymxpc2hlZCBzaXRlOiIgKippZHAubXljb21wYW55LmNvbSoqIHNvIHRoYXQgaGVhZGVyIEhvc3Qgd2lsbCBjb250YWluIHRoaXMgdmFsdWUuDQoNCk5vdyB3ZSBzaG91bGQgaGF2ZSBjaXJjdW12ZW50ZWQgb3VyIGZpcnN0IHByb2JsZW0gYnV0IGl0IHdpbGwgYmUgb2Ygbm8gaGVscCBzaW5jZSB0aGUgYGNoZWNrRW5kcG9pbnRVUklgICh0aHJvdWdoIGNhbGxpbmcgYGNvbXBhcmVFbmRwb2ludFVSSXNgLCBldGMpIHdpbGwgZG8gYSBzdHJpbmcgY29tcGFyaXNvbiB3aGljaCB3aWxsIG9idmlvdXNseSBmYWlsIGR1ZSB0byB0aGUgYWRkaXRpb25hbCBzdWJzdHJpbmcgIjo4NDQzIiBpbiB0aGUgYWN0dWFsIHJlY2VpdmVyRW5kcG9pbnQuDQoNCiMjIDg0NDMgUG9ydA0KDQpMb29raW5nIGF0IEpldHR5IGRvY3VtZW50YXRpb24gd2UgcXVpY2tseSBmaW5kIGZvdXIgc29sdXRpb25zIHRvIG92ZXJjb21lIHRoaXMgcHJvYmxlbToNCg0KKiBTdGFydCBKZXR0eSBhcyByb290IGFuZCBjaGFuZ2UgSFRQUFMgcG9ydCB0byA0NDM7IHdlIGRvIG5vdCB3YW50IHRvIGRvIHRoYXQgKHdvdWxkIHdlPykNCiogU29sdXRpb25zIGJhc2VkIG9uIGlwY2hhaW5zL2lwdGFibGUgd2hpY2ggaXMgbm90IGFjdHVhbGx5IHdoYXQgd2UgbmVlZDogdGhlcmUgd2lsbCBiZSBhbiBpbnRlcm5hbCBrZXJuZWwgcm91dGluZyBmcm9tIDQ0MyB0byA4NDQzIGFuZCBhdCB0aGUgZW5kIG9mIHRoZSBkYXkgd2Ugd2lsbCBnZXQgaW50byB0aGUgc2FtZSBwcm9ibGVtIGFzIG1lbnRpb25lZCBlYXJsaWVyDQoqIENvbmZpZ3VyaW5nIFNldFVJRCBKZXR0eSBmZWF0dXJlOyB0aGlzIGxvb2tzIGxpa2Ugc29sdmluZyBvdXIgcHJvYmxlbSBidXQgbmVlZHMgc29tZSBhZGRpdGlvbmFsIGtub3cgaG93IHdoaWNoIEkgd2FzIG5vdCB3aWxsaW5nIHRvIGFjcXVpcmUgKG5hdGl2ZSBjb21waWxpbmcsIHJ1bm5pbmcgdGhlIHNlcnZpY2UgYXMgcm9vdC4uLikNCg0KIyBUaGUgU29sdXRpb24NCg0KVGhlIHNvbHV0aW9uIHdhcyB0byB1c2UgSmV0dHkgcmV3cml0ZSBoYW5kbGVyIGNhcGFiaWxpdGllcyAoW2h0dHA6Ly93d3cuZWNsaXBzZS5vcmcvamV0dHkvZG9jdW1lbnRhdGlvbi9jdXJyZW50L3Jld3JpdGUtaGFuZGxlci5odG1sXShodHRwOi8vd3d3LmVjbGlwc2Uub3JnL2pldHR5L2RvY3VtZW50YXRpb24vY3VycmVudC9yZXdyaXRlLWhhbmRsZXIuaHRtbCkpLg0KDQpVbmZvcnR1bmF0ZWx5LCBub25lIG9mIHRoZSBvdXQgb2YgdGhlIGJveCBoYW5kbGVycyBwcm92aWRlZCB0aGUgcmVxdWlyZWQgZnVuY3Rpb25hbGl0eSBzbyBJIHF1aWNrbHkgd3JvdGUgb25lLg0KDQpCZWZvcmUgZ29pbmcgaW50byBpbnRvIHdyaXRpbmcvZGVwbG95aW5nIHRoZSBjdXN0b20gaGFuZGxlciB3ZSBoYXZlIHRvIHBlcmZvcm0gc29tZSBwcmVyZXF1aXNpdGVzOg0KDQoqIENoZWNrIHRvIHNlZSBpZiBgb3JnLmVjbGlwc2UuamV0dHkuc2VydmVyLlJlcXVlc3RgIHN1cHBvcnRzIHVwZGF0aW5nIGhvdHMgYW5kIHBvcnQgcHJvcGVydGllcyAoc2luY2Ugc3RhbmRhcmQgSHR0cFNlcnZsZXRSZXF1ZXN0IGRvZXMgbm90KQ0KKiBDaGVjayBob3cgdG8gc3BlY2lmeSB1cGRhdGVkIHBvcnQgc28gdGhhdCBpdCB3aWxsIG5vdCBiZSBpbmNsdWRlZCBpbiByZWNlaXZlckVuZHBvaW50IHN0cmluZw0KKiBDaGVjayBob3cgcmV3cml0ZSBoYW5kbGVyIGlzIGFjdGl2YXRlZA0KDQogDQpGaXJzdCBjcmVhdGUgYSBtYXZlbiBwcm9qZWN0IGhhdmluZyBqZXR0eS1yZXdyaXRlIGRlcGVuZGVuY3kgYW5kIGRvd25sb2FkIHNvdXJjZXMuLi4NCg0KYGBgeG1sDQogICAgICAgIDxkZXBlbmRlbmN5Pg0KICAgICAgICAgICAgPGdyb3VwSWQ+b3JnLmVjbGlwc2UuamV0dHk8L2dyb3VwSWQ+DQogICAgICAgICAgICA8YXJ0aWZhY3RJZD5qZXR0eS1yZXdyaXRlPC9hcnRpZmFjdElkPg0KICAgICAgICAgICAgPHZlcnNpb24+OS4zLjUudjIwMTUxMDEyPC92ZXJzaW9uPg0KICAgICAgICA8L2RlcGVuZGVuY3k+DQpgYGAgDQpGaXJzdCBpdGVtIGNoZWNrIHBhc3NlZCBzaW5jZSBsb29rcyBsaWtlIHdlIGNvdWxkIHVzZSANCmBgYGphdmENCiAgICAgICAgUmVxdWVzdCBqZXR0eVJlcXVlc3Q9KFJlcXVlc3QpIHJlcXVlc3Q7DQogICAgICAgIGpldHR5UmVxdWVzdC5nZXRNZXRhRGF0YSgpLmdldFVSSSgpLnNldEF1dGhvcml0eShob3N0LCBwb3J0KTsNCg0KYGBgDQpTZWNvbmQgaXRlbSBjaGVjayB3YXMgYWxzbyBxdWljazogYG9yZy5lY2xpcHNlLmpldHR5LnV0aWwuYXBwZW5kU2NoZW1lSG9zdFBvcnRgIHNob3dzIHRoYXQgaWYgc3BlY2lmeWluZyBkZWZhdWx0IHBvcnRzIChkZXBlbmRpbmcgb24gc2NoZW1hKSBvciAwIHRoZXkgd2lsbCBub3QgYmUgaW5jbHVkZWQuDQoNCg0KTGFzdCBpdGVtIHJlcXVpcmVzIHRoZSBmb2xsb3dpbmcgc3RlcHM6DQoNCkFkZCANCmBgYA0KLS1tb2R1bGU9cmV3cml0ZQ0KZXRjL3Jld3JpdGUtcnVsZXMueG1sDQpgYGANCmluIGAkSkVUVFlfQkFTRS9zdGFydC5pbmlgLCB3aGVyZSBgcmV3cml0ZS1ydWxlcy54bWxgIGNvbnRhaW5zIHRoZSBhY3R1YWwgcnVsZXMgYW5kIHNob3VsZCBiZSBsb2NhdGVkIGluICBgJEpFVFRZX0JBU0UvZXRjL3Jld3JpdGUtcnVsZXMueG1sYDoNCg0KYGBgeG1sDQo8P3htbCB2ZXJzaW9uPSIxLjAiPz4NCjwhRE9DVFlQRSBDb25maWd1cmUgUFVCTElDICItLy9KZXR0eS8vQ29uZmlndXJlLy9FTiIgImh0dHA6Ly93d3cuZWNsaXBzZS5vcmcvamV0dHkvY29uZmlndXJlXzlfMy5kdGQiPg0KDQo8IS0tID09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PSAtLT4NCjwhLS0gQ29uZmlndXJlIHRoZSBkZW1vcyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0tPg0KPCEtLSA9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0gLS0+DQo8Q29uZmlndXJlIGlkPSJTZXJ2ZXIiIGNsYXNzPSJvcmcuZWNsaXBzZS5qZXR0eS5zZXJ2ZXIuU2VydmVyIj4NCg0KICA8IS0tID09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0gLS0+DQogIDwhLS0gQWRkIHJld3JpdGUgcnVsZXMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtLT4NCiAgPCEtLSA9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09IC0tPg0KICA8UmVmIHJlZmlkPSJSZXdyaXRlIj4NCg0KICAgICAgPCEtLSBwcm90ZWN0IGZhdmljb24gaGFuZGxpbmcgLS0+DQogICAgICA8Q2FsbCBuYW1lPSJhZGRSdWxlIj4NCiAgICAgICAgPEFyZz4NCiAgICAgICAgICA8TmV3IGNsYXNzPSJyby50b3RhbHNvZnQuamV0dHkuaG9zdGhlYWRlcnJld3JpdGVyLkhvc3RIZWFkZXJSZXdyaXRlciI+DQogICAgICAgICAgICA8U2V0IG5hbWU9ImhlYWRlciI+SG9zdDwvU2V0Pg0KICAgICAgICAgICAgPFNldCBuYW1lPSJob3N0Ij5wcm94eS5teWNvbXBhbnkuY29tPC9TZXQ+DQogICAgICAgICAgICA8U2V0IG5hbWU9InBvcnQiPjA8L1NldD4NCiAgICAgICAgICAgIDxTZXQgbmFtZT0idGVybWluYXRpbmciPnRydWU8L1NldD4NCiAgICAgICAgICA8L05ldz4NCiAgICAgICAgPC9Bcmc+DQogICAgICA8L0NhbGw+DQogIDwvUmVmPg0KPC9Db25maWd1cmU+DQpgYGANCg0KSW4gb3JkZXIgdG8gYmUgc2VsZWN0aXZlIChpZiB3ZSBob3N0IGFsc28gb3RoZXIgYXBwbGljYXRpb25zIGluIHRoZSBzYW1lIEpldHR5IGluc3RhbmNlKSB3ZSBzaG91bGQgYWxzbyBhZGQgdGhlIGZpbHRlcmluZyBjb25kaXRpb24NCg0KYGBgeG1sDQo8U2V0IG5hbWU9ImhlYWRlclZhbHVlIj5pZHAubXljb21wYW55LmxvY2FsOjg0NDM8L1NldD4NCmBgYA0KDQpUaGUgYWN0dWFsIHJld3JpdGUgaGFuZGxlciBkZWZpbml0aW9uIGlzIGxvY2F0ZWQgaW4gYCRKRVRUWV9IT01FL2V0Yy9qZXR0eS1yZXdyaXRlLnhtbGAgYXMgY2FuIGJlIHNlZW4gYnkgaW5zcGVjdGlvbiB0aGUgYCRKRVRUWV9IT01FL21vZHVsZS9yZXdyaXRlLm1vZGAgbW9kdWxlIGRlZmluaXRpb24gZmlsZS4NCg0KUmVtYXJrOiBsb29raW5nIChhcyBhIG5ld2JpZSkgaW4gYGpldHR5LXJld3JpdGUueG1sYCBtaWdodCByYWlzZSBhIHF1ZXN0aW9uOiB3aGF0IGlzIGBvbGRoYW5kbGVyYCBhYm91dD8gVGhlIG5hbWluZyBpcyBtaXNsZWFkaW5nLCBpdCBzaG91bGQgYmUgc29tZXRoaW5nIGxpa2UgYGZpcnN0aGFuZGxlcmAgc2luY2UgaXQgcmVmZXJzIHRvIHRoZSBmaXJzdCBoYW5kbGVyIGluIHRoZSBjaGFpbiBvZiBKZXR0eSBoYW5kbGVycy4gQWZ0ZXIgSmV0dHkgYXBwbGllcyB0aGUgcmV3cml0ZSBoYW5kbGVyIGRlZmluaXRpb24gdGhpcyB3aWxsIGJlY29tZSB0aGUgZmlyc3QgaGFuZGxlciBpbiBjaGFpbi4NCg0KTm93IChzdXBwb3Npbmcgd2UgaGF2ZSB3cml0dGVuIG91ciBvd24gaGVhZGVyIHJ1bGUpIHdlIGhhdmUgdG8gZGVwbG95IGl0IGluIHRoZSBjbGFzc3BhdGgga25vd24gYnkgSmV0dHkuIFRvIGRvIHRoaXMgd2UgaGF2ZSB0byBhZGQgdGhlICoqZXh0KiogbW9kdWxlOg0KDQotLW1vZHVsZT1leHQNCmBgYA0KLS1tb2R1bGU9ZXh0DQpUaGUgZXh0IG1vZHVsZSB3aWxsIGVuYWJsZSB0aGUgbGliL2V4dC8qLmphciBsb2dpYy4NCg0KSWYgdGhpcyBtb2R1bGUgaXMgYWN0aXZhdGVkLCB0aGVuIGFsbCBqYXIgZmlsZXMgZm91bmQgaW4gdGhlIGxpYi9leHQvIHBhdGhzIHdpbGwgYmUgYXV0b21hdGljYWxseSBhZGRlZCB0byB0aGUgSmV0dHkgU2VydmVyIENsYXNzcGF0aC4NCmBgYA0KDQpBZnRlcndhcmRzIHdlIGNhbiBjb3B5IG91ciBqYXIgdG8gJEpFVFRZX0JBU0UvbGliL2V4dC8gYW5kIHJlc3RhcnQgSmV0dHkNCg0KKipJbXBvcnRhbnQgcmVtYXJrICoqOiBkdWUgdG8gYSBidWcgKG9taXNzaW9uKSBpbiB0aGUgbG9naWMgdGhhdCBhY3RpdmF0ZXMgcnVsZXMgaW5zaWRlIHRoZSByZXdyaXRlIGhhbmRsZXIgaWYgdGhlcmUgaXMgbm8gcnVsZSBkZWZpbmVkIGEgTnVsbFBvaW50ZXJFeGNlcHRpb24gd2lsbCBiZSB0aHJvd24gaW4gY2xhc3MgYG9yZy5lY2xpcHNlLmpldHR5LnJld3JpdGUuaGFuZGxlci5SdWxlQ29udGFpbmVyYCwgbWV0aG9kIGBhcHBseWA6DQoNCmBgYGphdmENCiAgICBwcm90ZWN0ZWQgU3RyaW5nIGFwcGx5KFN0cmluZyB0YXJnZXQsIEh0dHBTZXJ2bGV0UmVxdWVzdCByZXF1ZXN0LCBIdHRwU2VydmxldFJlc3BvbnNlIHJlc3BvbnNlKSB0aHJvd3MgSU9FeGNlcHRpb24NCiAgICB7DQogICAgICAgIGJvb2xlYW4gb3JpZ2luYWxfc2V0PV9vcmlnaW5hbFBhdGhBdHRyaWJ1dGU9PW51bGw7DQogICAgICAgICAgICAgICAgDQogICAgICAgIGZvciAoUnVsZSBydWxlIDogX3J1bGVzKQ0KICAgICAgICB7DQogICAgICAgICAgICBTdHJpbmcgYXBwbGllZD1ydWxlLm1hdGNoQW5kQXBwbHkodGFyZ2V0LHJlcXVlc3QsIHJlc3BvbnNlKTsNCiAgICAgICAgICAgIGlmIChhcHBsaWVkIT1udWxsKQ0KICAgICAgICAgICAgeyAgICAgICANCg0KYGBgDQpJZiBubyBydWxlcyBhcmUgZGVmaW5lZCB3ZSB3aWxsIGxhbmQgaGVyZSB3aXRoIGBfcnVsZXNgIGJlaW5nIG51bGwuLi4NCg0KIyBFcGlsb2d1ZQ0KDQpXdyBhcmUgbm90IGRvbmUgeWV0IGJ1dCBoYXZlIG1hZGUgc2lnbmlmaWNhbnQgcHJvZ3Jlc3MgaGF2aW5nIElkUCBzaWRlIHdvcmtpbmcuIEFzIHdlIGNhbiBhc3N1bWUsIHdlIHdpbGwgZmFjZSBzYW1lIHByb2JsZW0gb24gU1Agc2lkZSAod2UgZG8gbm90IHJlYWxseSB3YW50IHdlYiBzaXRlcyBleHBvc2VkIGRpcmVjdGx5IG9uIHRoZSBJbnRlcm5ldCkuDQoNClNvLCBuZXh0IHBvc3QgYWJvdXQgS2VudG9yIFNTTy4gSSB3b3VsZCBleHBlY3QgdG8gYXBwbHkgdGhlIHNvbHV0aW9uIHdpdGggZXh0ZXJuYWwvaW50ZXJuYWwgRE5TIGFuZCB1c2luZyA0NDMgZGVmYXVsdCBwb3J0IChXaW5kb3dzL0lJUyBsb29rcyBtb3JlIHBlcm1pc3NpdmUgaW4gdGhpcyBhcmVhKQ0KDQojIFRoZSBDb2RlDQoNCmBgYGphdmENCnBhY2thZ2UgY29tLm15Y29tcGFueS5qZXR0eS5ob3N0aGVhZGVycmV3cml0ZXI7DQoNCmltcG9ydCBqYXZhLmlvLklPRXhjZXB0aW9uOw0KaW1wb3J0IGphdmF4LnNlcnZsZXQuaHR0cC5IdHRwU2VydmxldFJlcXVlc3Q7DQppbXBvcnQgamF2YXguc2VydmxldC5odHRwLkh0dHBTZXJ2bGV0UmVzcG9uc2U7DQppbXBvcnQgb3JnLmVjbGlwc2UuamV0dHkucmV3cml0ZS5oYW5kbGVyLkhlYWRlclJ1bGU7DQppbXBvcnQgb3JnLmVjbGlwc2UuamV0dHkuc2VydmVyLlJlcXVlc3Q7DQoNCnB1YmxpYyBjbGFzcyBIb3N0SGVhZGVyUmV3cml0ZXIgZXh0ZW5kcyBIZWFkZXJSdWxlew0KICAgIFN0cmluZyBob3N0Ow0KICAgIGludCBwb3J0PTA7DQoNCiAgICBwdWJsaWMgU3RyaW5nIGdldEhvc3QoKSB7DQogICAgICAgIHJldHVybiBob3N0Ow0KICAgIH0NCiAgICBwdWJsaWMgdm9pZCBzZXRIb3N0KFN0cmluZyBob3N0KSB7DQogICAgICAgIHRoaXMuaG9zdCA9IGhvc3Q7DQogICAgfQ0KDQogICAgcHVibGljIGludCBnZXRQb3J0KCkgew0KICAgICAgICByZXR1cm4gcG9ydDsNCiAgICB9DQogICAgcHVibGljIHZvaWQgc2V0UG9ydChpbnQgcG9ydCkgew0KICAgICAgICB0aGlzLnBvcnQgPSBwb3J0Ow0KICAgIH0NCiAgICANCiAgICBAT3ZlcnJpZGUNCiAgICBwcm90ZWN0ZWQgU3RyaW5nIGFwcGx5KFN0cmluZyB0YXJnZXQsIFN0cmluZyB2YWx1ZSwgSHR0cFNlcnZsZXRSZXF1ZXN0IHJlcXVlc3QsIEh0dHBTZXJ2bGV0UmVzcG9uc2UgcmVzcG9uc2UpIHRocm93cyBJT0V4Y2VwdGlvbiB7DQogICAgICAgIFJlcXVlc3QgamV0dHlSZXF1ZXN0PShSZXF1ZXN0KSByZXF1ZXN0Ow0KICAgICAgICBqZXR0eVJlcXVlc3QuZ2V0TWV0YURhdGEoKS5nZXRVUkkoKS5zZXRBdXRob3JpdHkoaG9zdCwgcG9ydCk7DQogICAgICAgIHJldHVybiB0YXJnZXQ7DQogICAgfQ0KfQ0KYGBg

Monday, November 2, 2015

Shibboleth-We Are Not Done Yet-Part 1

First thing that popped out in my mind after basic installation/configuration worked was to check how could be the login page customized. But where to find that page?

The answer is simple: login.vm (idp-conf\src\main\resources\views\login.vm), a template based on Velocity. It was easy to find: look at the id assigned to password field (for instance), j_password and do a recursive search in sources directory.

Note: If changing main.css is necessary please notice that the UI part is packed as idp.war...

This is not enough; we want to understand how it works and where to look if we get into trouble.

To start with: a lot of Shibbolet functionality (including UI behaviour) is based on Spring Web Flow. The main configuration file (webflow-config.xml) can be found by looking into web.config.

    <servlet>
        <servlet-name>idp</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <init-param>
            <param-name>contextConfigLocation</param-name>
            <param-value>${idp.home}/system/conf/mvc-beans.xml, ${idp.home}/system/conf/webflow-config.xml</param-value>
        </init-param>
...

It defines application web-flows accessible through the registry (org.springframework.webflow.definition.registry.FlowDefinitionRegistryImpl, method getFlowDefinition, spring-webflow-2.4.1.RELEASE.jar).

The flow we are interested in is that with id="SAML2/Unsolicited/SSO". When we access HTTPS://<shibblolet-url>:<https-port>/idp/profile/SAML2/Unsolicited/SSO?providerId=<SP_entityID> DispatcherHandler will finally call org.springframework.webflow.mvc.servlet.FlowHandlerMapping, method getHandlerInternal, parameter HttpServletRequest request. Request properties are as follows:

  • contextPath: "/idp"
  • servletPath: "/profile"
  • pathInfo: "/SAML2/Unsolicited/SSO"

Based on pathInfo the web-flow with the matching id is activated. A breakpoint set in org.springframework.webflow.executor.FlowExecutorImpl, method launchExecution, after FlowDefinition flowDefinition = definitionLocator.getFlowDefinition(flowId); allows us to examine flowDefinition and see the states defined in the work-flow that is about to be executed.

Having a look in the flow definition (path="../system/flows/saml/saml2/sso-unsolicited-flow.xml") is quite confusing for the newbies: no actually flow description (according to what you will see in tutorials) but just two regular Spring beans definition: the point is that any such bean exposing an 'execute' method will be treated as a state-action and the methods will be called in the order of bean definition.

OK, but where comes in action our login.vm? The parent="saml2.sso.abstract" attribute in flow tag definition is the key: our flow inherits its definition from that flow (identified by this id in web-config.xml). Somehow (I haven't checked all the inheritance/definitions chain) the authn/Password sub-flow is activated, sub-flow that contains a view-state referring our template in the view attribute:

    <view-state id="DisplayUsernamePasswordPage" view="login">
        <on-render>
            <evaluate expression="environment" result="viewScope.environment" />
            <evaluate expression="opensamlProfileRequestContext" result="viewScope.profileRequestContext" />
            <evaluate expression="opensamlProfileRequestContext.getSubcontext(T(net.shibboleth.idp.authn.context.AuthenticationContext))" result="viewScope.authenticationContext" />
...

The transition <transition on="proceed" to="ExtractUsernamePasswordFromFormRequest"> is activated when the Login button is pressed (id='_eventId_proceed'-> proceed constant will become available to be evaluated by the transition condition based on the convention id=_eventId_... ).

Action state ExtractUsernamePasswordFromFormRequest will be executed:

    <action-state id="ExtractUsernamePasswordFromFormRequest">
        <evaluate expression="ExtractUsernamePasswordFromFormRequest" />
        <evaluate expression="'proceed'" />
    
        <!-- Let the validate action handle any problems later. -->        
        <transition to="ValidateUsernamePassword" />
    </action-state>

The <evaluate expression="ExtractUsernamePasswordFromFormRequest" /> has the role to call execute method of bean with id="ExtractUsernamePasswordFromFormRequest" ( class defined in module Shibboleth IdP :: Authentication Implementation) while <evaluate expression="'proceed'"/> just makes 'proceed' the current value to be evaluated by subsequent transitions (not actually used here but pattern used in may other places).

Finally we land in ValidateUsernamePasswordAgainstLDAP.execute method.

DQpGaXJzdCB0aGluZyB0aGF0IHBvcHBlZCBvdXQgaW4gbXkgbWluZCBhZnRlciBiYXNpYyBpbnN0YWxsYXRpb24vY29uZmlndXJhdGlvbiB3b3JrZWQgd2FzIHRvIGNoZWNrIGhvdyBjb3VsZCBiZSB0aGUgbG9naW4gcGFnZSBjdXN0b21pemVkLiAgQnV0IHdoZXJlIHRvIGZpbmQgdGhhdCBwYWdlPw0KDQpUaGUgYW5zd2VyIGlzIHNpbXBsZTogbG9naW4udm0gKGBpZHAtY29uZlxzcmNcbWFpblxyZXNvdXJjZXNcdmlld3NcbG9naW4udm1gKSwgYSB0ZW1wbGF0ZSBiYXNlZCBvbiBWZWxvY2l0eS4gSXQgd2FzIGVhc3kgdG8gZmluZDogbG9vayBhdCB0aGUgaWQgYXNzaWduZWQgdG8gcGFzc3dvcmQgZmllbGQgKGZvciBpbnN0YW5jZSksIGBqX3Bhc3N3b3JkYCAgYW5kIGRvIGEgcmVjdXJzaXZlIHNlYXJjaCBpbiAgc291cmNlcyBkaXJlY3RvcnkuDQoNCioqTm90ZSoqOiBJZiBjaGFuZ2luZyAqbWFpbi5jc3MqIGlzIG5lY2Vzc2FyeSBwbGVhc2Ugbm90aWNlIHRoYXQgdGhlIFVJIHBhcnQgaXMgcGFja2VkIGFzIGlkcC53YXIuLi4NCg0KVGhpcyBpcyBub3QgZW5vdWdoOyB3ZSB3YW50IHRvIHVuZGVyc3RhbmQgaG93IGl0IHdvcmtzIGFuZCB3aGVyZSB0byBsb29rIGlmIHdlIGdldCBpbnRvIHRyb3VibGUuDQoNClRvIHN0YXJ0IHdpdGg6IGEgbG90IG9mIFNoaWJib2xldCBmdW5jdGlvbmFsaXR5IChpbmNsdWRpbmcgVUkgYmVoYXZpb3VyKSBpcyBiYXNlZCBvbiBTcHJpbmcgV2ViIEZsb3cuIFRoZSBtYWluIGNvbmZpZ3VyYXRpb24gZmlsZSAoYHdlYmZsb3ctY29uZmlnLnhtbGApIGNhbiBiZSBmb3VuZCBieSBsb29raW5nIGludG8gYHdlYi5jb25maWdgLg0KYGBgeG1sDQogICAgPHNlcnZsZXQ+DQogICAgICAgIDxzZXJ2bGV0LW5hbWU+aWRwPC9zZXJ2bGV0LW5hbWU+DQogICAgICAgIDxzZXJ2bGV0LWNsYXNzPm9yZy5zcHJpbmdmcmFtZXdvcmsud2ViLnNlcnZsZXQuRGlzcGF0Y2hlclNlcnZsZXQ8L3NlcnZsZXQtY2xhc3M+DQogICAgICAgIDxpbml0LXBhcmFtPg0KICAgICAgICAgICAgPHBhcmFtLW5hbWU+Y29udGV4dENvbmZpZ0xvY2F0aW9uPC9wYXJhbS1uYW1lPg0KICAgICAgICAgICAgPHBhcmFtLXZhbHVlPiR7aWRwLmhvbWV9L3N5c3RlbS9jb25mL212Yy1iZWFucy54bWwsICR7aWRwLmhvbWV9L3N5c3RlbS9jb25mL3dlYmZsb3ctY29uZmlnLnhtbDwvcGFyYW0tdmFsdWU+DQogICAgICAgIDwvaW5pdC1wYXJhbT4NCi4uLg0KYGBgDQoNCkl0IGRlZmluZXMgYXBwbGljYXRpb24gd2ViLWZsb3dzIGFjY2Vzc2libGUgdGhyb3VnaCB0aGUgcmVnaXN0cnkgKGBvcmcuc3ByaW5nZnJhbWV3b3JrLndlYmZsb3cuZGVmaW5pdGlvbi5yZWdpc3RyeS5GbG93RGVmaW5pdGlvblJlZ2lzdHJ5SW1wbGAsIG1ldGhvZCBgZ2V0Rmxvd0RlZmluaXRpb25gLCBgc3ByaW5nLXdlYmZsb3ctMi40LjEuUkVMRUFTRS5qYXJgKS4NCg0KVGhlIGZsb3cgd2UgYXJlIGludGVyZXN0ZWQgaW4gaXMgdGhhdCB3aXRoIGBpZD0iU0FNTDIvVW5zb2xpY2l0ZWQvU1NPImAuIFdoZW4gd2UgYWNjZXNzIGBIVFRQUzovLzxzaGliYmxvbGV0LXVybD46PGh0dHBzLXBvcnQ+L2lkcC9wcm9maWxlL1NBTUwyL1Vuc29saWNpdGVkL1NTTz9wcm92aWRlcklkPTxTUF9lbnRpdHlJRD5gIGBEaXNwYXRjaGVySGFuZGxlcmAgd2lsbCBmaW5hbGx5IGNhbGwgYG9yZy5zcHJpbmdmcmFtZXdvcmsud2ViZmxvdy5tdmMuc2VydmxldC5GbG93SGFuZGxlck1hcHBpbmdgLCBtZXRob2QgYGdldEhhbmRsZXJJbnRlcm5hbGAsIHBhcmFtZXRlciBgSHR0cFNlcnZsZXRSZXF1ZXN0IHJlcXVlc3RgLiBSZXF1ZXN0IHByb3BlcnRpZXMgYXJlIGFzIGZvbGxvd3M6DQoNCiogY29udGV4dFBhdGg6ICIvaWRwIg0KKiBzZXJ2bGV0UGF0aDogIi9wcm9maWxlIg0KKiBwYXRoSW5mbzogIi9TQU1MMi9VbnNvbGljaXRlZC9TU08iDQoNCkJhc2VkIG9uIGBwYXRoSW5mb2AgdGhlIHdlYi1mbG93IHdpdGggdGhlIG1hdGNoaW5nIGlkIGlzIGFjdGl2YXRlZC4gQSBicmVha3BvaW50IHNldCBpbiBgb3JnLnNwcmluZ2ZyYW1ld29yay53ZWJmbG93LmV4ZWN1dG9yLkZsb3dFeGVjdXRvckltcGxgLCBtZXRob2QgYGxhdW5jaEV4ZWN1dGlvbmAsIGFmdGVyIGBGbG93RGVmaW5pdGlvbiBmbG93RGVmaW5pdGlvbiA9IGRlZmluaXRpb25Mb2NhdG9yLmdldEZsb3dEZWZpbml0aW9uKGZsb3dJZCk7YCBhbGxvd3MgdXMgdG8gZXhhbWluZSBgZmxvd0RlZmluaXRpb25gIGFuZCBzZWUgdGhlIHN0YXRlcyBkZWZpbmVkIGluIHRoZSB3b3JrLWZsb3cgdGhhdCBpcyBhYm91dCB0byBiZSBleGVjdXRlZC4NCg0KSGF2aW5nIGEgbG9vayBpbiB0aGUgZmxvdyBkZWZpbml0aW9uIChgcGF0aD0iLi4vc3lzdGVtL2Zsb3dzL3NhbWwvc2FtbDIvc3NvLXVuc29saWNpdGVkLWZsb3cueG1sImApIGlzIHF1aXRlIGNvbmZ1c2luZyBmb3IgdGhlIG5ld2JpZXM6IG5vIGFjdHVhbGx5IGZsb3cgZGVzY3JpcHRpb24gKGFjY29yZGluZyB0byB3aGF0IHlvdSB3aWxsIHNlZSBpbiB0dXRvcmlhbHMpIGJ1dCBqdXN0IHR3byByZWd1bGFyIFNwcmluZyBiZWFucyBkZWZpbml0aW9uOiB0aGUgcG9pbnQgaXMgdGhhdCBhbnkgc3VjaCBiZWFuIGV4cG9zaW5nIGFuICdleGVjdXRlJyBtZXRob2Qgd2lsbCBiZSB0cmVhdGVkIGFzIGEgc3RhdGUtYWN0aW9uIGFuZCB0aGUgbWV0aG9kcyB3aWxsIGJlIGNhbGxlZCBpbiB0aGUgb3JkZXIgb2YgYmVhbiBkZWZpbml0aW9uLg0KDQpPSywgYnV0IHdoZXJlIGNvbWVzIGluIGFjdGlvbiBvdXIgYGxvZ2luLnZtYD8gVGhlIGBwYXJlbnQ9InNhbWwyLnNzby5hYnN0cmFjdCJgIGF0dHJpYnV0ZSBpbiBmbG93IHRhZyBkZWZpbml0aW9uIGlzIHRoZSBrZXk6IG91ciBmbG93IGluaGVyaXRzIGl0cyBkZWZpbml0aW9uIGZyb20gdGhhdCBmbG93IChpZGVudGlmaWVkIGJ5IHRoaXMgaWQgaW4gYHdlYi1jb25maWcueG1sYCkuIFNvbWVob3cgKEkgaGF2ZW4ndCBjaGVja2VkIGFsbCB0aGUgaW5oZXJpdGFuY2UvZGVmaW5pdGlvbnMgY2hhaW4pIHRoZSBgYXV0aG4vUGFzc3dvcmRgIHN1Yi1mbG93IGlzIGFjdGl2YXRlZCwgc3ViLWZsb3cgdGhhdCBjb250YWlucyBhIGB2aWV3LXN0YXRlYCByZWZlcnJpbmcgb3VyIHRlbXBsYXRlIGluIHRoZSB2aWV3IGF0dHJpYnV0ZToNCmBgYHhtbA0KICAgIDx2aWV3LXN0YXRlIGlkPSJEaXNwbGF5VXNlcm5hbWVQYXNzd29yZFBhZ2UiIHZpZXc9ImxvZ2luIj4NCiAgICAgICAgPG9uLXJlbmRlcj4NCiAgICAgICAgICAgIDxldmFsdWF0ZSBleHByZXNzaW9uPSJlbnZpcm9ubWVudCIgcmVzdWx0PSJ2aWV3U2NvcGUuZW52aXJvbm1lbnQiIC8+DQogICAgICAgICAgICA8ZXZhbHVhdGUgZXhwcmVzc2lvbj0ib3BlbnNhbWxQcm9maWxlUmVxdWVzdENvbnRleHQiIHJlc3VsdD0idmlld1Njb3BlLnByb2ZpbGVSZXF1ZXN0Q29udGV4dCIgLz4NCiAgICAgICAgICAgIDxldmFsdWF0ZSBleHByZXNzaW9uPSJvcGVuc2FtbFByb2ZpbGVSZXF1ZXN0Q29udGV4dC5nZXRTdWJjb250ZXh0KFQobmV0LnNoaWJib2xldGguaWRwLmF1dGhuLmNvbnRleHQuQXV0aGVudGljYXRpb25Db250ZXh0KSkiIHJlc3VsdD0idmlld1Njb3BlLmF1dGhlbnRpY2F0aW9uQ29udGV4dCIgLz4NCi4uLg0KYGBgICANClRoZSB0cmFuc2l0aW9uIGA8dHJhbnNpdGlvbiBvbj0icHJvY2VlZCIgdG89IkV4dHJhY3RVc2VybmFtZVBhc3N3b3JkRnJvbUZvcm1SZXF1ZXN0Ij5gIGlzIGFjdGl2YXRlZCB3aGVuIHRoZSBMb2dpbiBidXR0b24gaXMgcHJlc3NlZCAoYGlkPSdfZXZlbnRJZF9wcm9jZWVkJ2AtXD4gYHByb2NlZWRgIGNvbnN0YW50IHdpbGwgYmVjb21lIGF2YWlsYWJsZSB0byBiZSBldmFsdWF0ZWQgYnkgdGhlIHRyYW5zaXRpb24gY29uZGl0aW9uIGJhc2VkIG9uIHRoZSBjb252ZW50aW9uIGBpZD1fZXZlbnRJZF8uLi5gICkuDQoNCkFjdGlvbiBzdGF0ZSBgRXh0cmFjdFVzZXJuYW1lUGFzc3dvcmRGcm9tRm9ybVJlcXVlc3RgIHdpbGwgYmUgZXhlY3V0ZWQ6DQpgYGB4bWwNCiAgICA8YWN0aW9uLXN0YXRlIGlkPSJFeHRyYWN0VXNlcm5hbWVQYXNzd29yZEZyb21Gb3JtUmVxdWVzdCI+DQogICAgICAgIDxldmFsdWF0ZSBleHByZXNzaW9uPSJFeHRyYWN0VXNlcm5hbWVQYXNzd29yZEZyb21Gb3JtUmVxdWVzdCIgLz4NCiAgICAgICAgPGV2YWx1YXRlIGV4cHJlc3Npb249Iidwcm9jZWVkJyIgLz4NCiAgICANCiAgICAgICAgPCEtLSBMZXQgdGhlIHZhbGlkYXRlIGFjdGlvbiBoYW5kbGUgYW55IHByb2JsZW1zIGxhdGVyLiAtLT4gICAgICAgIA0KICAgICAgICA8dHJhbnNpdGlvbiB0bz0iVmFsaWRhdGVVc2VybmFtZVBhc3N3b3JkIiAvPg0KICAgIDwvYWN0aW9uLXN0YXRlPg0KYGBgDQpUaGUgYDxldmFsdWF0ZSBleHByZXNzaW9uPSJFeHRyYWN0VXNlcm5hbWVQYXNzd29yZEZyb21Gb3JtUmVxdWVzdCIgLz5gIGhhcyB0aGUgcm9sZSB0byBjYWxsIGBleGVjdXRlYCBtZXRob2Qgb2YgYmVhbiB3aXRoIGBpZD0iRXh0cmFjdFVzZXJuYW1lUGFzc3dvcmRGcm9tRm9ybVJlcXVlc3QiYCAoIGNsYXNzIGRlZmluZWQgaW4gbW9kdWxlIGBTaGliYm9sZXRoIElkUCA6OiBBdXRoZW50aWNhdGlvbiBJbXBsZW1lbnRhdGlvbmApIHdoaWxlIGA8ZXZhbHVhdGUgZXhwcmVzc2lvbj0iJ3Byb2NlZWQnIi8+YCBqdXN0IG1ha2VzIGAncHJvY2VlZCdgIHRoZSBjdXJyZW50IHZhbHVlIHRvIGJlIGV2YWx1YXRlZCBieSBzdWJzZXF1ZW50IHRyYW5zaXRpb25zIChub3QgYWN0dWFsbHkgdXNlZCBoZXJlIGJ1dCBwYXR0ZXJuIHVzZWQgaW4gbWF5IG90aGVyIHBsYWNlcykuDQoNCkZpbmFsbHkgd2UgbGFuZCBpbiBgVmFsaWRhdGVVc2VybmFtZVBhc3N3b3JkQWdhaW5zdExEQVAuZXhlY3V0ZWAgbWV0aG9kLg==