Tuesday, March 1, 2016

Graylog series, part 1

Graylog Primer

This post applies to Graylog version 1.3.3.

Initial Server Install

This is straightforward: download the virtual appliance, import and start. That's it!

Graylog Collector

This post applies to version 0.4.2.

  • Download graylog-collector-0.4.2.zip archieve and explode it in target directory
  • Rename collector.conf.example to collector.conf; Modify it according to your needs (there are plenty of samples in Graylog documentation)
  • Install/start the service running bin\graylog-collector-service.bat install GraylogCollector and bin\graylog-collector-service.bat start GraylogCollector

Now we have a server side problem: by default there are two pre-installed inputs :appliance-gelf-udp and appliance-syslog-udp while the client Collector supports (for GELF at least) only TCP. So we have to create another input according to our configuration (port mainly).

If everything worked according to the plan we should see now in the Search section our events and every second a nginx sourced event (very annoying).

Server insights

Startup scripts

/opt/graylog/service contains symbolic links to all stratup directories. In /etc/init/graylog-runsvdir.conf there is executed exec /opt/graylog/embedded/bin/runsvdir-start which in turn runs exec env - PATH=$PATH runsvdir -P /opt/graylog/service 'log:.

Every directory pointed by the symbolic links mentioned before contains a run script. For instance in /opt/graylog/sv/graylog-server/ we have

#!/bin/sh
exec 2>&1

umask 077

if [ -f "/opt/graylog/embedded/share/graylog/installation-source.sh" ]; then
    . "/opt/graylog/embedded/share/graylog/installation-source.sh"
fi

export JAVA_HOME=/opt/graylog/embedded/jre
export GRAYLOG_SERVER_JAVA_OPTS="-Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"

# check if mongodb is up
timeout 600 bash -c "until curl -s http://127.0.0.1:27017; do sleep 1; done"
exec chpst -P -U graylog -u graylog /opt/graylog/embedded/bin/authbind $JAVA_HOME/bin/java $GRAYLOG_SERVER_JAVA_OPTS -jar -Dlog4j.configuration=file:///opt/graylog/conf/log4j.xml -Djava.library.path=/opt/graylog/server/lib/sigar/ -Dgraylog2.installation_source=${GRAYLOG_INSTALLATION_SOURCE:=unknown} /opt/graylog/server/graylog.jar server -f /opt/graylog/conf/graylog.conf

Logging

Logging is started through log\run. See svloggelfd documentation on github as well as linux svlogd manual. Notice that UDP is used by svloggelfd.

Actual logs are saved in /var/log/graylog/... according to svlogd rules while svloggelfd sends all data to Graylog UDP input.

Sample log\run content:

#!/bin/sh
exec /opt/graylog/embedded/bin/svloggelfd -H 127.0.0.1:12201 -s graylog-server -e | svlogd -tt /var/log/graylog/server

Disabling nginx Loging forward to Graylog

Just cahnge /opt/graylog/sv/nginx/log/run to:

#!/bin/sh
#exec /opt/graylog/embedded/bin/svloggelfd -H 127.0.0.1:12201 -s nginx -e | svlogd -tt /var/log/graylog/nginx
exec svlogd -tt /var/log/graylog/nginx

and restart Graylog (see below)

Controllig Graylog from command line

graylog-ctl status
graylog-ctl stop
graylog-ctl start

Timezone

Use sudo date to change the default settings for date/time (UTC+0).

Changing timezone :

  • Edit /etc/timezone and change to the desired timezone (like Europe/Bucharest)
  • Run sudo dpkg-reconfigure --frontend noninteractive tzdata

This is not enough; Graylog web interface will still disply events with time in Etc/UTC timezone.

So:

  • Check the startup script for graylog-web (semething like exec chpst -P -U graylog -u graylog /opt/graylog/web/bin/graylog-web-interface -Dconfig.file=/opt/graylog/conf/graylog-web-interface.conf -Dhttp.port=9000 -Dhttp.address=0.0.0.0 -Dpidfile.path=/var/opt/graylog/web.pid -Dlogger.file=/opt/graylog/conf/web-logger.xml
  • Now edit the configuration file (/opt/graylog/conf/graylog-web-interface.conf) and change the timezone property according to your needs
  • This might not be enough: admin user has a fixed UTC timezone configuration so you will have to create a new user and assign the desired timezone or none (so the default just configured before will be used)

Debugging

This is not about bugs but just inspecting the implementation code to clarify things not captured in official documentation

Server

  • Change startup script (/opt/graylog/sv/graylog-server/run) to
#!/bin/sh
exec 2>&1

umask 077

if [ -f "/opt/graylog/embedded/share/graylog/installation-source.sh" ]; then
    . "/opt/graylog/embedded/share/graylog/installation-source.sh"
fi

export JAVA_HOME=/opt/graylog/embedded/jre
# export GRAYLOG_SERVER_JAVA_OPTS="-Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"
export GRAYLOG_SERVER_JAVA_OPTS="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=4242 -Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"

# check if mongodb is up
timeout 600 bash -c "until curl -s http://127.0.0.1:27017; do sleep 1; done"
exec chpst -P -U graylog -u graylog /opt/graylog/embedded/bin/authbind $JAVA_HOME/bin/java $GRAYLOG_SERVER_JAVA_OPTS -jar -Dlog4j.configuration=file:///opt/graylog/conf/log4j.xml -Djava.library.path=/opt/graylog/server/lib/sigar/ -Dgraylog2.installation_source=${GRAYLOG_INSTALLATION_SOURCE:=unknown} /opt/graylog/server/graylog.jar server -f /opt/graylog/conf/graylog.conf
  • Download tag 1.3.3 of graylog-server from github
  • Open pom.xml in your favorite Java IDE
  • Attach to <graylog-server-fqdn>, port 4242
  • Happy inspecting!

Web interface

Debugging web interface is a little bit tricky since it is written in Scala-Play without a maven based project, so:

  • Change the startup script (/opt/graylog/sv/graylog-web/run) to:
#!/bin/sh
exec 2>&1

umask 077
export JAVA_HOME=/opt/graylog/embedded/jre

rm -f /var/opt/graylog/web.pid
exec chpst -P -U graylog -u graylog /opt/graylog/web/bin/graylog-web-interface -jvm-debug 4243 -Dconfig.file=/opt/graylog/conf/graylog-web-interface.conf -Dhttp.port=9000 -Dhttp.address=0.0.0.0 -Dpidfile.path=/var/opt/graylog/web.pid -Dlogger.file=/opt/graylog/conf/web-logger.xml
  • Notice the -jvm-debug 4243 extra parameter
  • Configure your favourite Java IDE to use the source path also from graylog2-web-interface-1.3.3\app (download tag 1.3.3 of deprecated web interface sources)
  • Some functionality refers to classes in graylog2-rest-client module of graylog server project (like org.graylog2.restclient.lib.DateTools containing method getUserTimeZone).
  • Search is performed in renderSearch method of controllers.SearchController class.
  • Render template is graylog2-web-interface-1.3.3\app\views\search\index.scala.html
  • Java script dealing with rendering dates according to the user's timezone is done in app\assets\javascripts\moment-helper.js while gl2UserTimeZone is defined in app\views\partials\navbar.scala.html
IyBHcmF5bG9nIFByaW1lcg0KVGhpcyBwb3N0IGFwcGxpZXMgdG8gR3JheWxvZyB2ZXJzaW9uIDEuMy4zLg0KDQojIyBJbml0aWFsIFNlcnZlciBJbnN0YWxsDQoNClRoaXMgaXMgc3RyYWlnaHRmb3J3YXJkOiBkb3dubG9hZCB0aGUgdmlydHVhbCBhcHBsaWFuY2UsIGltcG9ydCBhbmQgc3RhcnQuIFRoYXQncyBpdCENCg0KIyMgR3JheWxvZyBDb2xsZWN0b3INCg0KVGhpcyBwb3N0IGFwcGxpZXMgdG8gdmVyc2lvbiAwLjQuMi4NCg0KKiBEb3dubG9hZCBncmF5bG9nLWNvbGxlY3Rvci0wLjQuMi56aXAgYXJjaGlldmUgYW5kIGV4cGxvZGUgaXQgaW4gdGFyZ2V0IGRpcmVjdG9yeQ0KKiBSZW5hbWUgYGNvbGxlY3Rvci5jb25mLmV4YW1wbGVgIHRvIGBjb2xsZWN0b3IuY29uZmA7IE1vZGlmeSBpdCBhY2NvcmRpbmcgdG8geW91ciBuZWVkcyAodGhlcmUgYXJlIHBsZW50eSBvZiBzYW1wbGVzIGluIEdyYXlsb2cgZG9jdW1lbnRhdGlvbikNCiogSW5zdGFsbC9zdGFydCB0aGUgc2VydmljZSBydW5uaW5nIGBiaW5cZ3JheWxvZy1jb2xsZWN0b3Itc2VydmljZS5iYXQgaW5zdGFsbCBHcmF5bG9nQ29sbGVjdG9yYCBhbmQgYGJpblxncmF5bG9nLWNvbGxlY3Rvci1zZXJ2aWNlLmJhdCBzdGFydCBHcmF5bG9nQ29sbGVjdG9yYA0KDQpOb3cgd2UgaGF2ZSBhIHNlcnZlciBzaWRlIHByb2JsZW06IGJ5IGRlZmF1bHQgdGhlcmUgYXJlIHR3byBwcmUtaW5zdGFsbGVkIGlucHV0cyA6YGFwcGxpYW5jZS1nZWxmLXVkcGAgYW5kIGBhcHBsaWFuY2Utc3lzbG9nLXVkcGAgd2hpbGUgdGhlIGNsaWVudCBDb2xsZWN0b3Igc3VwcG9ydHMgKGZvciBHRUxGIGF0IGxlYXN0KSBvbmx5IFRDUC4gU28gd2UgaGF2ZSB0byBjcmVhdGUgYW5vdGhlciBpbnB1dCBhY2NvcmRpbmcgdG8gb3VyIGNvbmZpZ3VyYXRpb24gKGBwb3J0YCBtYWlubHkpLg0KDQpJZiBldmVyeXRoaW5nIHdvcmtlZCBhY2NvcmRpbmcgdG8gdGhlIHBsYW4gd2Ugc2hvdWxkIHNlZSBub3cgaW4gdGhlIFNlYXJjaCBzZWN0aW9uIG91ciBldmVudHMgYW5kIGV2ZXJ5IHNlY29uZCBhIGBuZ2lueGAgc291cmNlZCBldmVudCAodmVyeSBhbm5veWluZykuDQoNCiMjIFNlcnZlciBpbnNpZ2h0cw0KDQojIyMgU3RhcnR1cCBzY3JpcHRzDQoNCmAvb3B0L2dyYXlsb2cvc2VydmljZWAgY29udGFpbnMgc3ltYm9saWMgbGlua3MgdG8gYWxsIHN0cmF0dXAgZGlyZWN0b3JpZXMuIEluIGAvZXRjL2luaXQvZ3JheWxvZy1ydW5zdmRpci5jb25mYCB0aGVyZSBpcyBleGVjdXRlZCBgZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3J1bnN2ZGlyLXN0YXJ0YCB3aGljaCBpbiB0dXJuIHJ1bnMgYGV4ZWMgZW52IC0gUEFUSD0kUEFUSCBydW5zdmRpciAtUCAvb3B0L2dyYXlsb2cvc2VydmljZSAnbG9nOmAuDQoNCkV2ZXJ5IGRpcmVjdG9yeSBwb2ludGVkIGJ5IHRoZSBzeW1ib2xpYyBsaW5rcyBtZW50aW9uZWQgYmVmb3JlIGNvbnRhaW5zIGEgYHJ1bmAgc2NyaXB0LiBGb3IgaW5zdGFuY2UgaW4gYC9vcHQvZ3JheWxvZy9zdi9ncmF5bG9nLXNlcnZlci9gIHdlIGhhdmUNCg0KYGBgYmFzaA0KIyEvYmluL3NoDQpleGVjIDI+JjENCg0KdW1hc2sgMDc3DQoNCmlmIFsgLWYgIi9vcHQvZ3JheWxvZy9lbWJlZGRlZC9zaGFyZS9ncmF5bG9nL2luc3RhbGxhdGlvbi1zb3VyY2Uuc2giIF07IHRoZW4NCiAgICAuICIvb3B0L2dyYXlsb2cvZW1iZWRkZWQvc2hhcmUvZ3JheWxvZy9pbnN0YWxsYXRpb24tc291cmNlLnNoIg0KZmkNCg0KZXhwb3J0IEpBVkFfSE9NRT0vb3B0L2dyYXlsb2cvZW1iZWRkZWQvanJlDQpleHBvcnQgR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTPSItWG1zMWcgLVhteDE1MDBtIC1YWDpOZXdSYXRpbz0xIC1YWDpQZXJtU2l6ZT0xMjhtIC1YWDpNYXhQZXJtU2l6ZT0yNTZtIC1zZXJ2ZXIgLVhYOitSZXNpemVUTEFCIC1YWDorVXNlQ29uY01hcmtTd2VlcEdDIC1YWDorQ01TQ29uY3VycmVudE1URW5hYmxlZCAtWFg6K0NNU0NsYXNzVW5sb2FkaW5nRW5hYmxlZCAtWFg6K1VzZVBhck5ld0dDIC1YWDotT21pdFN0YWNrVHJhY2VJbkZhc3RUaHJvdyINCg0KIyBjaGVjayBpZiBtb25nb2RiIGlzIHVwDQp0aW1lb3V0IDYwMCBiYXNoIC1jICJ1bnRpbCBjdXJsIC1zIGh0dHA6Ly8xMjcuMC4wLjE6MjcwMTc7IGRvIHNsZWVwIDE7IGRvbmUiDQpleGVjIGNocHN0IC1QIC1VIGdyYXlsb2cgLXUgZ3JheWxvZyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL2F1dGhiaW5kICRKQVZBX0hPTUUvYmluL2phdmEgJEdSQVlMT0dfU0VSVkVSX0pBVkFfT1BUUyAtamFyIC1EbG9nNGouY29uZmlndXJhdGlvbj1maWxlOi8vL29wdC9ncmF5bG9nL2NvbmYvbG9nNGoueG1sIC1EamF2YS5saWJyYXJ5LnBhdGg9L29wdC9ncmF5bG9nL3NlcnZlci9saWIvc2lnYXIvIC1EZ3JheWxvZzIuaW5zdGFsbGF0aW9uX3NvdXJjZT0ke0dSQVlMT0dfSU5TVEFMTEFUSU9OX1NPVVJDRTo9dW5rbm93bn0gL29wdC9ncmF5bG9nL3NlcnZlci9ncmF5bG9nLmphciBzZXJ2ZXIgLWYgL29wdC9ncmF5bG9nL2NvbmYvZ3JheWxvZy5jb25mDQpgYGANCg0KIyMjIExvZ2dpbmcNCg0KTG9nZ2luZyBpcyBzdGFydGVkIHRocm91Z2ggYGxvZ1xydW5gLiBTZWUgYHN2bG9nZ2VsZmRgIGRvY3VtZW50YXRpb24gb24gZ2l0aHViIGFzIHdlbGwgYXMgbGludXggYHN2bG9nZGAgbWFudWFsLiBOb3RpY2UgdGhhdCBVRFAgaXMgdXNlZCBieSBgc3Zsb2dnZWxmZGAuDQoNCkFjdHVhbCBsb2dzIGFyZSBzYXZlZCBpbiBgL3Zhci9sb2cvZ3JheWxvZy8uLi5gIGFjY29yZGluZyB0byBgc3Zsb2dkYCBydWxlcyB3aGlsZSBgc3Zsb2dnZWxmZGAgc2VuZHMgYWxsIGRhdGEgdG8gR3JheWxvZyBVRFAgaW5wdXQuDQoNClNhbXBsZSBgbG9nXHJ1bmAgY29udGVudDoNCmBgYGJhc2gNCiMhL2Jpbi9zaA0KZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3N2bG9nZ2VsZmQgLUggMTI3LjAuMC4xOjEyMjAxIC1zIGdyYXlsb2ctc2VydmVyIC1lIHwgc3Zsb2dkIC10dCAvdmFyL2xvZy9ncmF5bG9nL3NlcnZlcg0KYGBgDQoNCiMjIyBEaXNhYmxpbmcgbmdpbnggTG9naW5nIGZvcndhcmQgdG8gR3JheWxvZw0KDQpKdXN0IGNhaG5nZSBgL29wdC9ncmF5bG9nL3N2L25naW54L2xvZy9ydW5gIHRvOg0KYGBgYmFzaA0KIyEvYmluL3NoDQojZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3N2bG9nZ2VsZmQgLUggMTI3LjAuMC4xOjEyMjAxIC1zIG5naW54IC1lIHwgc3Zsb2dkIC10dCAvdmFyL2xvZy9ncmF5bG9nL25naW54DQpleGVjIHN2bG9nZCAtdHQgL3Zhci9sb2cvZ3JheWxvZy9uZ2lueA0KYGBgDQphbmQgcmVzdGFydCBHcmF5bG9nIChzZWUgYmVsb3cpDQoNCiMjIyBDb250cm9sbGlnIEdyYXlsb2cgZnJvbSBjb21tYW5kIGxpbmUNCg0KYGBgYmFzaA0KZ3JheWxvZy1jdGwgc3RhdHVzDQpncmF5bG9nLWN0bCBzdG9wDQpncmF5bG9nLWN0bCBzdGFydA0KYGBgDQoNCiMjIyBUaW1lem9uZQ0KDQpVc2UgYHN1ZG8gZGF0ZWAgdG8gY2hhbmdlIHRoZSBkZWZhdWx0IHNldHRpbmdzIGZvciBkYXRlL3RpbWUgKFVUQyswKS4NCg0KQ2hhbmdpbmcgdGltZXpvbmUgOg0KKiBFZGl0IC9ldGMvdGltZXpvbmUgYW5kIGNoYW5nZSB0byB0aGUgZGVzaXJlZCB0aW1lem9uZSAobGlrZSBFdXJvcGUvQnVjaGFyZXN0KQ0KKiBSdW4gc3VkbyBkcGtnLXJlY29uZmlndXJlIC0tZnJvbnRlbmQgbm9uaW50ZXJhY3RpdmUgdHpkYXRhDQoNClRoaXMgaXMgbm90IGVub3VnaDsgR3JheWxvZyB3ZWIgaW50ZXJmYWNlIHdpbGwgc3RpbGwgZGlzcGx5IGV2ZW50cyB3aXRoIHRpbWUgaW4gRXRjL1VUQyB0aW1lem9uZS4NCg0KU286DQoqIENoZWNrIHRoZSBzdGFydHVwIHNjcmlwdCBmb3IgZ3JheWxvZy13ZWIgKHNlbWV0aGluZyBsaWtlIGBleGVjIGNocHN0IC1QIC1VIGdyYXlsb2cgLXUgZ3JheWxvZyAvb3B0L2dyYXlsb2cvd2ViL2Jpbi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UgLURjb25maWcuZmlsZT0vb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UuY29uZiAtRGh0dHAucG9ydD05MDAwIC1EaHR0cC5hZGRyZXNzPTAuMC4wLjAgLURwaWRmaWxlLnBhdGg9L3Zhci9vcHQvZ3JheWxvZy93ZWIucGlkIC1EbG9nZ2VyLmZpbGU9L29wdC9ncmF5bG9nL2NvbmYvd2ViLWxvZ2dlci54bWxgDQoqIE5vdyBlZGl0IHRoZSBjb25maWd1cmF0aW9uIGZpbGUgKGAvb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UuY29uZmApIGFuZCBjaGFuZ2UgdGhlIHRpbWV6b25lIHByb3BlcnR5IGFjY29yZGluZyB0byB5b3VyIG5lZWRzDQoqIFRoaXMgbWlnaHQgbm90IGJlIGVub3VnaDogYWRtaW4gdXNlciBoYXMgYSBmaXhlZCBVVEMgdGltZXpvbmUgY29uZmlndXJhdGlvbiBzbyB5b3Ugd2lsbCBoYXZlIHRvIGNyZWF0ZSBhIG5ldyB1c2VyIGFuZCBhc3NpZ24gdGhlIGRlc2lyZWQgdGltZXpvbmUgb3Igbm9uZSAoc28gdGhlIGRlZmF1bHQganVzdCBjb25maWd1cmVkIGJlZm9yZSB3aWxsIGJlIHVzZWQpDQoNCiMjIyBEZWJ1Z2dpbmcNCg0KVGhpcyBpcyBub3QgYWJvdXQgYnVncyBidXQganVzdCBpbnNwZWN0aW5nIHRoZSBpbXBsZW1lbnRhdGlvbiBjb2RlIHRvIGNsYXJpZnkgdGhpbmdzIG5vdCBjYXB0dXJlZCBpbiBvZmZpY2lhbCBkb2N1bWVudGF0aW9uDQoNCiMjIyMgU2VydmVyDQoNCiogQ2hhbmdlIHN0YXJ0dXAgc2NyaXB0IChgL29wdC9ncmF5bG9nL3N2L2dyYXlsb2ctc2VydmVyL3J1bmApIHRvDQpgYGBiYXNoDQojIS9iaW4vc2gNCmV4ZWMgMj4mMQ0KDQp1bWFzayAwNzcNCg0KaWYgWyAtZiAiL29wdC9ncmF5bG9nL2VtYmVkZGVkL3NoYXJlL2dyYXlsb2cvaW5zdGFsbGF0aW9uLXNvdXJjZS5zaCIgXTsgdGhlbg0KICAgIC4gIi9vcHQvZ3JheWxvZy9lbWJlZGRlZC9zaGFyZS9ncmF5bG9nL2luc3RhbGxhdGlvbi1zb3VyY2Uuc2giDQpmaQ0KDQpleHBvcnQgSkFWQV9IT01FPS9vcHQvZ3JheWxvZy9lbWJlZGRlZC9qcmUNCiMgZXhwb3J0IEdSQVlMT0dfU0VSVkVSX0pBVkFfT1BUUz0iLVhtczFnIC1YbXgxNTAwbSAtWFg6TmV3UmF0aW89MSAtWFg6UGVybVNpemU9MTI4bSAtWFg6TWF4UGVybVNpemU9MjU2bSAtc2VydmVyIC1YWDorUmVzaXplVExBQiAtWFg6K1VzZUNvbmNNYXJrU3dlZXBHQyAtWFg6K0NNU0NvbmN1cnJlbnRNVEVuYWJsZWQgLVhYOitDTVNDbGFzc1VubG9hZGluZ0VuYWJsZWQgLVhYOitVc2VQYXJOZXdHQyAtWFg6LU9taXRTdGFja1RyYWNlSW5GYXN0VGhyb3ciDQpleHBvcnQgR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTPSItWGRlYnVnIC1Ybm9hZ2VudCAtWHJ1bmpkd3A6dHJhbnNwb3J0PWR0X3NvY2tldCxzZXJ2ZXI9eSxzdXNwZW5kPW4sYWRkcmVzcz00MjQyIC1YbXMxZyAtWG14MTUwMG0gLVhYOk5ld1JhdGlvPTEgLVhYOlBlcm1TaXplPTEyOG0gLVhYOk1heFBlcm1TaXplPTI1Nm0gLXNlcnZlciAtWFg6K1Jlc2l6ZVRMQUIgLVhYOitVc2VDb25jTWFya1N3ZWVwR0MgLVhYOitDTVNDb25jdXJyZW50TVRFbmFibGVkIC1YWDorQ01TQ2xhc3NVbmxvYWRpbmdFbmFibGVkIC1YWDorVXNlUGFyTmV3R0MgLVhYOi1PbWl0U3RhY2tUcmFjZUluRmFzdFRocm93Ig0KDQojIGNoZWNrIGlmIG1vbmdvZGIgaXMgdXANCnRpbWVvdXQgNjAwIGJhc2ggLWMgInVudGlsIGN1cmwgLXMgaHR0cDovLzEyNy4wLjAuMToyNzAxNzsgZG8gc2xlZXAgMTsgZG9uZSINCmV4ZWMgY2hwc3QgLVAgLVUgZ3JheWxvZyAtdSBncmF5bG9nIC9vcHQvZ3JheWxvZy9lbWJlZGRlZC9iaW4vYXV0aGJpbmQgJEpBVkFfSE9NRS9iaW4vamF2YSAkR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTIC1qYXIgLURsb2c0ai5jb25maWd1cmF0aW9uPWZpbGU6Ly8vb3B0L2dyYXlsb2cvY29uZi9sb2c0ai54bWwgLURqYXZhLmxpYnJhcnkucGF0aD0vb3B0L2dyYXlsb2cvc2VydmVyL2xpYi9zaWdhci8gLURncmF5bG9nMi5pbnN0YWxsYXRpb25fc291cmNlPSR7R1JBWUxPR19JTlNUQUxMQVRJT05fU09VUkNFOj11bmtub3dufSAvb3B0L2dyYXlsb2cvc2VydmVyL2dyYXlsb2cuamFyIHNlcnZlciAtZiAvb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLmNvbmYNCmBgYA0KKiBEb3dubG9hZCB0YWcgMS4zLjMgb2YgZ3JheWxvZy1zZXJ2ZXIgZnJvbSBnaXRodWINCiogT3BlbiBwb20ueG1sIGluIHlvdXIgZmF2b3JpdGUgSmF2YSBJREUNCiogQXR0YWNoIHRvIDxncmF5bG9nLXNlcnZlci1mcWRuPiwgcG9ydCA0MjQyDQoqIEhhcHB5IGluc3BlY3RpbmchDQoNCiMjIyMgV2ViIGludGVyZmFjZQ0KDQpEZWJ1Z2dpbmcgd2ViIGludGVyZmFjZSBpcyBhIGxpdHRsZSBiaXQgdHJpY2t5IHNpbmNlIGl0IGlzIHdyaXR0ZW4gaW4gU2NhbGEtUGxheSB3aXRob3V0IGEgbWF2ZW4gYmFzZWQgcHJvamVjdCwgc286DQoNCiogQ2hhbmdlIHRoZSBzdGFydHVwIHNjcmlwdCAoYC9vcHQvZ3JheWxvZy9zdi9ncmF5bG9nLXdlYi9ydW5gKSB0bzoNCmBgYGJhc2gNCiMhL2Jpbi9zaA0KZXhlYyAyPiYxDQoNCnVtYXNrIDA3Nw0KZXhwb3J0IEpBVkFfSE9NRT0vb3B0L2dyYXlsb2cvZW1iZWRkZWQvanJlDQoNCnJtIC1mIC92YXIvb3B0L2dyYXlsb2cvd2ViLnBpZA0KZXhlYyBjaHBzdCAtUCAtVSBncmF5bG9nIC11IGdyYXlsb2cgL29wdC9ncmF5bG9nL3dlYi9iaW4vZ3JheWxvZy13ZWItaW50ZXJmYWNlIC1qdm0tZGVidWcgNDI0MyAtRGNvbmZpZy5maWxlPS9vcHQvZ3JheWxvZy9jb25mL2dyYXlsb2ctd2ViLWludGVyZmFjZS5jb25mIC1EaHR0cC5wb3J0PTkwMDAgLURodHRwLmFkZHJlc3M9MC4wLjAuMCAtRHBpZGZpbGUucGF0aD0vdmFyL29wdC9ncmF5bG9nL3dlYi5waWQgLURsb2dnZXIuZmlsZT0vb3B0L2dyYXlsb2cvY29uZi93ZWItbG9nZ2VyLnhtbA0KYGBgDQoqIE5vdGljZSB0aGUgYC1qdm0tZGVidWcgNDI0M2AgZXh0cmEgcGFyYW1ldGVyDQoqIENvbmZpZ3VyZSB5b3VyIGZhdm91cml0ZSBKYXZhIElERSB0byB1c2UgdGhlIHNvdXJjZSBwYXRoIGFsc28gZnJvbSBgZ3JheWxvZzItd2ViLWludGVyZmFjZS0xLjMuM1xhcHBgIChkb3dubG9hZCB0YWcgMS4zLjMgb2YgZGVwcmVjYXRlZCB3ZWIgaW50ZXJmYWNlIHNvdXJjZXMpDQoqIFNvbWUgZnVuY3Rpb25hbGl0eSByZWZlcnMgdG8gY2xhc3NlcyBpbiBgZ3JheWxvZzItcmVzdC1jbGllbnRgIG1vZHVsZSBvZiBncmF5bG9nIHNlcnZlciBwcm9qZWN0ICAobGlrZSBgb3JnLmdyYXlsb2cyLnJlc3RjbGllbnQubGliLkRhdGVUb29sc2AgY29udGFpbmluZyBtZXRob2QgYGdldFVzZXJUaW1lWm9uZWApLg0KKiBTZWFyY2ggaXMgcGVyZm9ybWVkIGluIGByZW5kZXJTZWFyY2hgIG1ldGhvZCBvZiBgY29udHJvbGxlcnMuU2VhcmNoQ29udHJvbGxlcmAgY2xhc3MuDQoqIFJlbmRlciB0ZW1wbGF0ZSBpcyBgZ3JheWxvZzItd2ViLWludGVyZmFjZS0xLjMuM1xhcHBcdmlld3Ncc2VhcmNoXGluZGV4LnNjYWxhLmh0bWxgDQoqIEphdmEgc2NyaXB0IGRlYWxpbmcgd2l0aCByZW5kZXJpbmcgZGF0ZXMgYWNjb3JkaW5nIHRvIHRoZSB1c2VyJ3MgdGltZXpvbmUgaXMgZG9uZSBpbiBgYXBwXGFzc2V0c1xqYXZhc2NyaXB0c1xtb21lbnQtaGVscGVyLmpzYCB3aGlsZSBgZ2wyVXNlclRpbWVab25lYCBpcyBkZWZpbmVkIGluIGBhcHBcdmlld3NccGFydGlhbHNcbmF2YmFyLnNjYWxhLmh0bWxg
Posted by at

No comments :

Post a Comment

Subscribe to: Post Comments ( Atom )