Tuesday, October 11, 2016

Install PostgeSQL on Ubuntu 16.04

Based on https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-16-04

Install PosgreSQL:

sudo apt-get update
sudo apt-get install postgresql postgresql-contrib

Find out where configuration files are stored:

sudo -u postgres psql # use \q to exit

show config_file; # somethis like /etc/postgresql/9.5/main...
\q

Cchange in posgresql.config
- #listen_addresses = 'localhost' # what IP address(es) to listen on;
- to listen_addresses = '*' # what IP address(es) to listen on;
Add in pg_hba.conf
- host all all 10.0.0.0/8 md5 Change through psql the password for database login postgres:
- \password postgres
Restart postgresql:
- exit (to logout from postgres user session)
- systemctl restart postgreql

Install Wildfly 10.1 on Ubuntu Server 16.04

Install Wildfly 10.1 on Ubuntu 16.04

Be sure to use the same Java version as the one used as compilation target of generated WAR.

Run the following script:

sudo apt-get update
apt install software-properties-common
add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

cd /tmp
wget http://download.jboss.org/wildfly/10.1.0.Final/wildfly-10.1.0.Final.tar.gz
tar -xzvf wildfly-10.1.0.Final.tar.gz
mv wildfly-10.1.0.Final wildfly
cp -R  wildfly /opt/

groupadd -r wildfly
useradd -r -g wildfly -d /opt/wildfly -s /sbin/nologin wildfly
chown -R wildfly:wildfly /opt/wildfly

cd wildfly/docs/contrib/scripts/systemd

mkdir /etc/wildfly
cp wildfly.conf /etc/wildfly/
cp wildfly.service /etc/systemd/system/
cp launch.sh /opt/wildfly/bin/
chmod +x /opt/wildfly/bin/launch.sh

# do not launch wildfly through standalone.sh->it will screw up file permissions

systemctl start wildfly.service
systemctl enable wildfly.service

Watch out for CRLF EndOfLine (should be just LFs)
Check that Wildfly is running through systemctl status wildfly

In order to check from outside current VM/physical machine that Wildfly was started change standalone.xml content so that:

        <interface name="public">
            <any-address/>
        </interface>

There might be required some other changes to the default distribution. Besure that wildfly user is owner all added items (chown -R wildfly:wildfly /opt/wildfly/modules)

Add EclipseLink Support

Download eclipselink.jar to modules\system\layers\base\org\eclipse\persistence\main
Change module.xml so that:

    <resources>
        <resource-root path="jipijapa-eclipselink-10.1.0.Final.jar"/>
        <resource-root path="eclipselink.jar">
            <filter>
                <exclude path="javax/**" />
            </filter>
        </resource-root>
    </resources>

Add SQL Server support

Create modules\system\layers\base\com\microsoft\sqlserver\jdbc\main
Download sqljdbc4.jar
Create module.xml:

<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.3" name="com.microsoft.sqlserver.jdbc">
    <resources>
        <resource-root path="sqljdbc4.jar"/>
    </resources>
    <dependencies>
        <module name="javax.api"/>
        <module name="javax.transaction.api"/>
    </dependencies>
</module>

Add PostgreSQL Support

Create modules\system\layers\base\org\postgresql\main
Download latest JDBC PostgreSQL driver
Create module.xml(change driver name acordingly):

<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.3" name="org.postgresql">
    <resources>
        <!-- Make sure this matches the name of the JAR you are installing -->
        <resource-root path="postgresql-9.4-1202.jdbc4.jar"/>
    </resources>
    <dependencies>
        <module name="javax.api"/>
        <module name="javax.transaction.api"/>
        <module name="javax.servlet.api" optional="true"/>
    </dependencies>
</module>

Restart Wildfly...

Velocity Note

If some of the used components in generated WAR use Velocity you will encounter the following error in server.log (during hot deployment or Wildfly restart): Error configuring AvalonLogSystem : java.io.FileNotFoundException: /velocity.log (Permission denied). This is due to the fact that velocity.properties (as found in velocity jar, path org\apache\velocity\runtime\defaults\ states:

runtime.log = velocity.log

The quick solution is to create an empty velocity.log file in the root of your file system with full access (?) for wildfly user.

IyBJbnN0YWxsIFdpbGRmbHkgMTAuMSBvbiBVYnVudHUgMTYuMDQNCg0KKipCZSBzdXJlIHRvIHVzZSB0aGUgc2FtZSBKYXZhIHZlcnNpb24gYXMgdGhlIG9uZSB1c2VkIGFzIGNvbXBpbGF0aW9uIHRhcmdldCBvZiBnZW5lcmF0ZWQgV0FSLioqDQoNClJ1biB0aGUgZm9sbG93aW5nIHNjcmlwdDoNCmBgYA0Kc3VkbyBhcHQtZ2V0IHVwZGF0ZQ0KYXB0IGluc3RhbGwgc29mdHdhcmUtcHJvcGVydGllcy1jb21tb24NCmFkZC1hcHQtcmVwb3NpdG9yeSBwcGE6d2VidXBkOHRlYW0vamF2YQ0Kc3VkbyBhcHQtZ2V0IHVwZGF0ZQ0Kc3VkbyBhcHQtZ2V0IGluc3RhbGwgb3JhY2xlLWphdmE4LWluc3RhbGxlcg0KDQpjZCAvdG1wDQp3Z2V0IGh0dHA6Ly9kb3dubG9hZC5qYm9zcy5vcmcvd2lsZGZseS8xMC4xLjAuRmluYWwvd2lsZGZseS0xMC4xLjAuRmluYWwudGFyLmd6DQp0YXIgLXh6dmYgd2lsZGZseS0xMC4xLjAuRmluYWwudGFyLmd6DQptdiB3aWxkZmx5LTEwLjEuMC5GaW5hbCB3aWxkZmx5DQpjcCAtUiAgd2lsZGZseSAvb3B0Lw0KDQpncm91cGFkZCAtciB3aWxkZmx5DQp1c2VyYWRkIC1yIC1nIHdpbGRmbHkgLWQgL29wdC93aWxkZmx5IC1zIC9zYmluL25vbG9naW4gd2lsZGZseQ0KY2hvd24gLVIgd2lsZGZseTp3aWxkZmx5IC9vcHQvd2lsZGZseQ0KDQpjZCB3aWxkZmx5L2RvY3MvY29udHJpYi9zY3JpcHRzL3N5c3RlbWQNCg0KbWtkaXIgL2V0Yy93aWxkZmx5DQpjcCB3aWxkZmx5LmNvbmYgL2V0Yy93aWxkZmx5Lw0KY3Agd2lsZGZseS5zZXJ2aWNlIC9ldGMvc3lzdGVtZC9zeXN0ZW0vDQpjcCBsYXVuY2guc2ggL29wdC93aWxkZmx5L2Jpbi8NCmNobW9kICt4IC9vcHQvd2lsZGZseS9iaW4vbGF1bmNoLnNoDQoNCiMgZG8gbm90IGxhdW5jaCB3aWxkZmx5IHRocm91Z2ggc3RhbmRhbG9uZS5zaC0+aXQgd2lsbCBzY3JldyB1cCBmaWxlIHBlcm1pc3Npb25zDQoNCnN5c3RlbWN0bCBzdGFydCB3aWxkZmx5LnNlcnZpY2UNCnN5c3RlbWN0bCBlbmFibGUgd2lsZGZseS5zZXJ2aWNlDQpgYGANCg0KLSBXYXRjaCBvdXQgZm9yIENSTEYgRW5kT2ZMaW5lIChzaG91bGQgYmUganVzdCBMRnMpDQotIENoZWNrIHRoYXQgV2lsZGZseSBpcyBydW5uaW5nIHRocm91Z2ggYHN5c3RlbWN0bCBzdGF0dXMgd2lsZGZseWANCg0KSW4gb3JkZXIgdG8gY2hlY2sgZnJvbSBvdXRzaWRlIGN1cnJlbnQgVk0vcGh5c2ljYWwgbWFjaGluZSB0aGF0IFdpbGRmbHkgd2FzIHN0YXJ0ZWQgY2hhbmdlIHN0YW5kYWxvbmUueG1sIGNvbnRlbnQgc28gdGhhdDoNCmBgYHhtbA0KICAgICAgICA8aW50ZXJmYWNlIG5hbWU9InB1YmxpYyI+DQogICAgICAgICAgICA8YW55LWFkZHJlc3MvPg0KICAgICAgICA8L2ludGVyZmFjZT4NCmBgYA0KVGhlcmUgbWlnaHQgYmUgcmVxdWlyZWQgc29tZSBvdGhlciBjaGFuZ2VzIHRvIHRoZSBkZWZhdWx0IGRpc3RyaWJ1dGlvbi4gQmVzdXJlIHRoYXQgd2lsZGZseSB1c2VyIGlzIG93bmVyIGFsbCBhZGRlZCBpdGVtcyAoYGNob3duIC1SIHdpbGRmbHk6d2lsZGZseSAvb3B0L3dpbGRmbHkvbW9kdWxlc2ApDQoNCiMjIEFkZCBFY2xpcHNlTGluayBTdXBwb3J0DQoNCi0gRG93bmxvYWQgZWNsaXBzZWxpbmsuamFyIHRvIGBtb2R1bGVzXHN5c3RlbVxsYXllcnNcYmFzZVxvcmdcZWNsaXBzZVxwZXJzaXN0ZW5jZVxtYWluYA0KLSBDaGFuZ2UgYG1vZHVsZS54bWxgIHNvIHRoYXQ6DQpgYGB4bWwNCiAgICA8cmVzb3VyY2VzPg0KICAgICAgICA8cmVzb3VyY2Utcm9vdCBwYXRoPSJqaXBpamFwYS1lY2xpcHNlbGluay0xMC4xLjAuRmluYWwuamFyIi8+DQoJCTxyZXNvdXJjZS1yb290IHBhdGg9ImVjbGlwc2VsaW5rLmphciI+DQoJCQk8ZmlsdGVyPg0KCQkJCTxleGNsdWRlIHBhdGg9ImphdmF4LyoqIiAvPg0KCQkJPC9maWx0ZXI+DQoJCTwvcmVzb3VyY2Utcm9vdD4NCiAgICA8L3Jlc291cmNlcz4NCmBgYA0KDQojIyBBZGQgU1FMIFNlcnZlciBzdXBwb3J0DQoNCi0gQ3JlYXRlIGBtb2R1bGVzXHN5c3RlbVxsYXllcnNcYmFzZVxjb21cbWljcm9zb2Z0XHNxbHNlcnZlclxqZGJjXG1haW5gDQotIERvd25sb2FkIGBzcWxqZGJjNC5qYXJgDQotIENyZWF0ZSBgbW9kdWxlLnhtbGA6DQpgYGB4bWwNCjw/eG1sIHZlcnNpb249IjEuMCIgZW5jb2Rpbmc9IlVURi04Ij8+DQo8bW9kdWxlIHhtbG5zPSJ1cm46amJvc3M6bW9kdWxlOjEuMyIgbmFtZT0iY29tLm1pY3Jvc29mdC5zcWxzZXJ2ZXIuamRiYyI+DQogICAgPHJlc291cmNlcz4NCiAgICAgICAgPHJlc291cmNlLXJvb3QgcGF0aD0ic3FsamRiYzQuamFyIi8+DQogICAgPC9yZXNvdXJjZXM+DQogICAgPGRlcGVuZGVuY2llcz4NCiAgICAgICAgPG1vZHVsZSBuYW1lPSJqYXZheC5hcGkiLz4NCiAgICAgICAgPG1vZHVsZSBuYW1lPSJqYXZheC50cmFuc2FjdGlvbi5hcGkiLz4NCiAgICA8L2RlcGVuZGVuY2llcz4NCjwvbW9kdWxlPg0KYGBgIA0KDQojIyBBZGQgUG9zdGdyZVNRTCBTdXBwb3J0DQoNCg0KLSBDcmVhdGUgYG1vZHVsZXNcc3lzdGVtXGxheWVyc1xiYXNlXG9yZ1xwb3N0Z3Jlc3FsXG1haW5gDQotIERvd25sb2FkIGxhdGVzdCBKREJDIFBvc3RncmVTUUwgZHJpdmVyDQotIENyZWF0ZSBgbW9kdWxlLnhtbGAoY2hhbmdlIGRyaXZlciBuYW1lIGFjb3JkaW5nbHkpOg0KYGBgeG1sDQo8P3htbCB2ZXJzaW9uPSIxLjAiIGVuY29kaW5nPSJVVEYtOCI/Pg0KPG1vZHVsZSB4bWxucz0idXJuOmpib3NzOm1vZHVsZToxLjMiIG5hbWU9Im9yZy5wb3N0Z3Jlc3FsIj4NCiAgICA8cmVzb3VyY2VzPg0KICAgICAgICA8IS0tIE1ha2Ugc3VyZSB0aGlzIG1hdGNoZXMgdGhlIG5hbWUgb2YgdGhlIEpBUiB5b3UgYXJlIGluc3RhbGxpbmcgLS0+DQogICAgICAgIDxyZXNvdXJjZS1yb290IHBhdGg9InBvc3RncmVzcWwtOS40LTEyMDIuamRiYzQuamFyIi8+DQogICAgPC9yZXNvdXJjZXM+DQogICAgPGRlcGVuZGVuY2llcz4NCiAgICAgICAgPG1vZHVsZSBuYW1lPSJqYXZheC5hcGkiLz4NCiAgICAgICAgPG1vZHVsZSBuYW1lPSJqYXZheC50cmFuc2FjdGlvbi5hcGkiLz4NCiAgICAgICAgPG1vZHVsZSBuYW1lPSJqYXZheC5zZXJ2bGV0LmFwaSIgb3B0aW9uYWw9InRydWUiLz4NCiAgICA8L2RlcGVuZGVuY2llcz4NCjwvbW9kdWxlPg0KYGBgDQoNCiMjIFJlc3RhcnQgV2lsZGZseS4uLg0KDQojIyBWZWxvY2l0eSBOb3RlDQoNCklmIHNvbWUgb2YgdGhlIHVzZWQgY29tcG9uZW50cyBpbiBnZW5lcmF0ZWQgV0FSIHVzZSBWZWxvY2l0eSB5b3Ugd2lsbCBlbmNvdW50ZXIgdGhlIGZvbGxvd2luZyBlcnJvciBpbiBzZXJ2ZXIubG9nIChkdXJpbmcgaG90IGRlcGxveW1lbnQgb3IgV2lsZGZseSByZXN0YXJ0KTogYEVycm9yIGNvbmZpZ3VyaW5nIEF2YWxvbkxvZ1N5c3RlbSA6IGphdmEuaW8uRmlsZU5vdEZvdW5kRXhjZXB0aW9uOiAvdmVsb2NpdHkubG9nIChQZXJtaXNzaW9uIGRlbmllZClgLiBUaGlzIGlzIGR1ZSB0byB0aGUgZmFjdCB0aGF0IHZlbG9jaXR5LnByb3BlcnRpZXMgKGFzIGZvdW5kIGluIHZlbG9jaXR5IGphciwgcGF0aCBgb3JnXGFwYWNoZVx2ZWxvY2l0eVxydW50aW1lXGRlZmF1bHRzXGAgc3RhdGVzOg0KYGBgDQpydW50aW1lLmxvZyA9IHZlbG9jaXR5LmxvZw0KYGBgDQpUaGUgcXVpY2sgc29sdXRpb24gaXMgdG8gY3JlYXRlIGFuIGVtcHR5IGB2ZWxvY2l0eS5sb2dgIGZpbGUgaW4gdGhlIHJvb3Qgb2YgeW91ciBmaWxlIHN5c3RlbSB3aXRoIGZ1bGwgYWNjZXNzICg/KSBmb3Igd2lsZGZseSB1c2VyLg0KIA==

Thursday, October 6, 2016

Graylog2 & MongoDB

Graylog uses an instance of MongoDB as a configuration repository. Browsing data can be done (for instance) by using Robomango:

Install Robomango, free edition (Robomango)
Configure a connection, providing Address (IP or FQDN) and Port (27017)
Happy browsing!

Graylog2 Rules

Running server side rules that drop/enrich messages is quite simple:

edit /opt/graylog/conf/graylog.conf by adding the following line rules_file = /opt/graylog/conf/rules.drl (there is special empty section starting with the comment # Drools Rule File (Use to rewrite incoming log messages)
add /opt/graylog/conf/rules.drl file with the appropriate content
restart graylog (sudo graylog-ctl start, etc.)

Sample rules file

import org.graylog2.plugin.Message
import java.util.regex.Matcher
import java.util.regex.Pattern

rule "WildFly10"
  when
      m : Message( getField("source_id") == "WildFly10" )
  then
      Matcher matcher = Pattern.compile("^\\d{4}-\\d{2}-\\d{2}\\s\\d{2}:\\d{2}:\\d{2},\\d{3}\\s(\\w+)").matcher(m.getMessage());
      if (matcher.find()) {
         m.addField("event-level", matcher.group(1));
      }
end

Graylog2 Collector

The new approach

Server side controlled configuration

Staring with Graylog 2 the a new collector approach (Graylog Collector Sidecar) is proposed, based on wrapping NXLog.

Installation is straightforward, as indicated in online documents (sample for Windows, a later post might share Ubuntu experience).

Install NXLog:

Download NXLog Windows installation kit NXLog downloads
Run the installer
Run (as an administrator) "C:\Program Files (x86)\nxlog\nxlog.exe" -u; this will un-install nxlog as a service but will keep binaries

Now download and install latest collector-sidecar binaries (Collector-Sidecar releases)

Before installing/starting the collector as a service:

On the server administration site go to System->Collectors->Manage Configurations
Add a configuration with a meaningful name
- Add one or more relevant tags
- Add an Output (by choosing from already configured Inputs)
- Add one or more inputs; for file type inputs you can check the multi-line option and provide a RegEx for the start pattern, like /^\d{2}\.\d{2}\.\d{4}/ (notice enclosing slashes!)

Go back to the monitored computer and edit C:\Program Files (x86)\graylog\collector-sidecar\collector-sidecar.yaml:

update server_url
update to a meaningful name the node_id
add one of the tags previously defined in the tags list
install and start the service (run as administrator):
- "C:\Program Files (x86)\graylog\collector-sidecar\graylog-collector-sidecar.exe" -service install
- "C:\Program Files (x86)\graylog\collector-sidecar\graylog-collector-sidecar.exe" -service start

*Service can be started/stopped also from Task manager (Services tab, collector-sidecar Name) or from Services Manager (Graylog collector sidecar Display name).

Now we should have automatically created, based on the previously defined configuration, the nxlog configuration file C:\Program Files (x86)\graylog\collector-sidecar\generated\nxlog.conf and in administration site the collector should be listed in System->Collectors section.

Please notice that if there is some error in the yaml configuration file the service will not start and no error message will be logged (it happened to me if I commented the tags list, see next section)

Locally controlled configuration

Under certain conditions you might need to use advanced NXLog features which are not supported by the server side configuration. You can do the following trick:

Start with the approach described in the previous section and check that everything goes fine; at this moment we should have a valid NXLog configuration file
Move the configuration file in another directory and change the configuration_path value to that file
Remove all valid tags and add a dummy one that does not exists in server side configuration; do not remove the tags entry or leave the list empty since the collector will not start and no error message will be logged
Restart the collector-sidecar service
If everything works as expected, read carefully the NXLog manual and do your own enhancements to the configuration
Restart the collector-sidecar service

I applied the approach because I needed an aditional message field (Exec $source_id = 'WildFly10';) source_id for one of the inputs, like shown below.

<Input 576d1d3d659fb107f722db38>
    Module im_file
    File "C:\Java\wildfly-10.0.0.Final\standalone\log\server.log"
    PollInterval 1
    SavePos True
    ReadFromLast True
    Recursive False
    RenameCheck False
    Exec $FileName = file_name(); # Send file name with each message
    Exec $source_id = 'WildFly10';
    InputType 576d1d3d659fb107f722db38-multiline
</Input>

The full yaml configuration file content:

server_url: http://graylog.mycompany.local:12900
tls_skip_verify: false
node_id: graylog-collector-sidecar-devsrv
collector_id: file:C:\Program Files (x86)\graylog\collector-sidecar\collector-id
tags:
    - undefined
log_path: C:\Program Files (x86)\graylog\collector-sidecar
update_interval: 10
backends:
    - name: nxlog
      enabled: true
      binary_path: C:\Program Files (x86)\nxlog\nxlog.exe
#      configuration_path: C:\Program Files (x86)\graylog\collector-sidecar\generated\nxlog.conf
      configuration_path: C:\Program Files (x86)\graylog\collector-sidecar\conf\nxlog.conf

As an alternate approach (I haven't tested but it should work) is to go for the first option (server side controlled configuration) but instead of defining Outputs/Inputs just place all nxlog.conf content in the default nxlog snippet, in the Windows section. Be sure you have in sync the collector side client_id with the one in the nxlog.conf content, Output section (see Exec $gl2_source_collector = '0324bd79-b9a9-458e-9977-d0eecbd2d347';):

<Output 576cf426659fb107f722b011>
    Module om_tcp
    Host graylog.mycompany.local
    Port 12201
    OutputType  GELF_TCP
    Exec $short_message = $raw_event; # Avoids truncation of the short_message field.
    Exec $gl2_source_collector = '0324bd79-b9a9-458e-9977-d0eecbd2d347';
    Exec $Hostname = hostname_fqdn();
</Output>

Duplicate configuration on another server

Once we have a collector configured we can duplicate easily the installation on another one:

Perform the NXLog installation steps
Install the collector-sidecar (just run the setup)
Copy yaml and nxlog.conf files on the target server
Change the yaml to reflect the right node_id
Change the content of the collector-id with some unique string (a GUID?)
Change the nxlog.conf Output tag to reflect the collector-id value

IyBUaGUgbmV3IGFwcHJvYWNoDQoNCiMjIFNlcnZlciBzaWRlIGNvbnRyb2xsZWQgY29uZmlndXJhdGlvbg0KDQpTdGFyaW5nIHdpdGggR3JheWxvZyAyIHRoZSBhIG5ldyBjb2xsZWN0b3IgYXBwcm9hY2ggKGBHcmF5bG9nIENvbGxlY3RvciBTaWRlY2FyYCkgaXMgcHJvcG9zZWQsIGJhc2VkIG9uIHdyYXBwaW5nIGBOWExvZ2AuDQoNCkluc3RhbGxhdGlvbiBpcyBzdHJhaWdodGZvcndhcmQsIGFzIGluZGljYXRlZCBpbiBvbmxpbmUgZG9jdW1lbnRzIChzYW1wbGUgZm9yIFdpbmRvd3MsIGEgbGF0ZXIgcG9zdCBtaWdodCBzaGFyZSBVYnVudHUgZXhwZXJpZW5jZSkuDQoNCkluc3RhbGwgTlhMb2c6DQoNCi0gRG93bmxvYWQgTlhMb2cgV2luZG93cyBpbnN0YWxsYXRpb24ga2l0IFtOWExvZyBkb3dubG9hZHNdKGh0dHBzOi8vbnhsb2cub3JnL3Byb2R1Y3RzL254bG9nLWNvbW11bml0eS1lZGl0aW9uL2Rvd25sb2FkICJOWExvZyBkb3dubG9hZHMiKQ0KLSBSdW4gdGhlIGluc3RhbGxlcg0KLSBSdW4gKGFzIGFuIGFkbWluaXN0cmF0b3IpIGAiQzpcUHJvZ3JhbSBGaWxlcyAoeDg2KVxueGxvZ1xueGxvZy5leGUiIC11YDsgdGhpcyB3aWxsIHVuLWluc3RhbGwgYG54bG9nYCBhcyBhIHNlcnZpY2UgYnV0IHdpbGwga2VlcCBiaW5hcmllcw0KDQpOb3cgZG93bmxvYWQgYW5kIGluc3RhbGwgbGF0ZXN0IGNvbGxlY3Rvci1zaWRlY2FyIGJpbmFyaWVzIChbQ29sbGVjdG9yLVNpZGVjYXIgcmVsZWFzZXNdKGh0dHBzOi8vZ2l0aHViLmNvbS9HcmF5bG9nMi9jb2xsZWN0b3Itc2lkZWNhci9yZWxlYXNlcykpDQoNCkJlZm9yZSBpbnN0YWxsaW5nL3N0YXJ0aW5nIHRoZSBjb2xsZWN0b3IgYXMgYSBzZXJ2aWNlOg0KDQotIE9uIHRoZSBzZXJ2ZXIgYWRtaW5pc3RyYXRpb24gc2l0ZSBnbyB0byBTeXN0ZW0tPkNvbGxlY3RvcnMtPk1hbmFnZSBDb25maWd1cmF0aW9ucw0KLSBBZGQgYSBjb25maWd1cmF0aW9uIHdpdGggYSBtZWFuaW5nZnVsIG5hbWUNCgktIEFkZCBvbmUgb3IgbW9yZSByZWxldmFudCB0YWdzDQoJLSBBZGQgYW4gT3V0cHV0IChieSBjaG9vc2luZyBmcm9tIGFscmVhZHkgY29uZmlndXJlZCBJbnB1dHMpDQoJLSBBZGQgb25lIG9yIG1vcmUgaW5wdXRzOyBmb3IgZmlsZSB0eXBlIGlucHV0cyB5b3UgY2FuIGNoZWNrIHRoZSBtdWx0aS1saW5lIG9wdGlvbiBhbmQgcHJvdmlkZSBhIFJlZ0V4IGZvciB0aGUgc3RhcnQgcGF0dGVybiwgbGlrZSBgL15cZHsyfVwuXGR7Mn1cLlxkezR9L2AgKG5vdGljZSBlbmNsb3Npbmcgc2xhc2hlcyEpDQoNCkdvIGJhY2sgdG8gdGhlIG1vbml0b3JlZCBjb21wdXRlciBhbmQgZWRpdCBgQzpcUHJvZ3JhbSBGaWxlcyAoeDg2KVxncmF5bG9nXGNvbGxlY3Rvci1zaWRlY2FyXGNvbGxlY3Rvci1zaWRlY2FyLnlhbWxgOg0KDQotIHVwZGF0ZSBzZXJ2ZXJfdXJsDQotIHVwZGF0ZSB0byBhIG1lYW5pbmdmdWwgbmFtZSB0aGUgbm9kZV9pZA0KLSBhZGQgb25lIG9mIHRoZSB0YWdzIHByZXZpb3VzbHkgZGVmaW5lZCBpbiB0aGUgdGFncyBsaXN0DQotIGluc3RhbGwgYW5kIHN0YXJ0IHRoZSBzZXJ2aWNlIChydW4gYXMgYWRtaW5pc3RyYXRvcik6DQoJLSBgIkM6XFByb2dyYW0gRmlsZXMgKHg4NilcZ3JheWxvZ1xjb2xsZWN0b3Itc2lkZWNhclxncmF5bG9nLWNvbGxlY3Rvci1zaWRlY2FyLmV4ZSIgLXNlcnZpY2UgaW5zdGFsbGANCgktIGAiQzpcUHJvZ3JhbSBGaWxlcyAoeDg2KVxncmF5bG9nXGNvbGxlY3Rvci1zaWRlY2FyXGdyYXlsb2ctY29sbGVjdG9yLXNpZGVjYXIuZXhlIiAtc2VydmljZSBzdGFydGANCg0KKlNlcnZpY2UgY2FuIGJlIHN0YXJ0ZWQvc3RvcHBlZCBhbHNvIGZyb20gVGFzayBtYW5hZ2VyIChTZXJ2aWNlcyB0YWIsIGBjb2xsZWN0b3Itc2lkZWNhcmAgTmFtZSkgb3IgZnJvbSBTZXJ2aWNlcyBNYW5hZ2VyIChgR3JheWxvZyBjb2xsZWN0b3Igc2lkZWNhcmAgRGlzcGxheSBuYW1lKS4NCg0KTm93IHdlIHNob3VsZCBoYXZlIGF1dG9tYXRpY2FsbHkgY3JlYXRlZCwgYmFzZWQgb24gdGhlIHByZXZpb3VzbHkgZGVmaW5lZCBjb25maWd1cmF0aW9uLCB0aGUgbnhsb2cgY29uZmlndXJhdGlvbiBmaWxlIGBDOlxQcm9ncmFtIEZpbGVzICh4ODYpXGdyYXlsb2dcY29sbGVjdG9yLXNpZGVjYXJcZ2VuZXJhdGVkXG54bG9nLmNvbmZgIGFuZCBpbiBhZG1pbmlzdHJhdGlvbiBzaXRlIHRoZSBjb2xsZWN0b3Igc2hvdWxkIGJlIGxpc3RlZCBpbiBTeXN0ZW0tPkNvbGxlY3RvcnMgc2VjdGlvbi4NCg0KUGxlYXNlIG5vdGljZSB0aGF0IGlmIHRoZXJlIGlzIHNvbWUgZXJyb3IgaW4gdGhlIHlhbWwgY29uZmlndXJhdGlvbiBmaWxlIHRoZSBzZXJ2aWNlIHdpbGwgbm90IHN0YXJ0IGFuZCBubyBlcnJvciBtZXNzYWdlIHdpbGwgYmUgbG9nZ2VkIChpdCBoYXBwZW5lZCB0byBtZSBpZiBJIGNvbW1lbnRlZCB0aGUgdGFncyBsaXN0LCBzZWUgbmV4dCBzZWN0aW9uKQ0KDQojIyBMb2NhbGx5IGNvbnRyb2xsZWQgY29uZmlndXJhdGlvbg0KDQpVbmRlciBjZXJ0YWluIGNvbmRpdGlvbnMgeW91IG1pZ2h0IG5lZWQgdG8gdXNlIGFkdmFuY2VkIE5YTG9nIGZlYXR1cmVzIHdoaWNoIGFyZSBub3Qgc3VwcG9ydGVkIGJ5IHRoZSBzZXJ2ZXIgc2lkZSBjb25maWd1cmF0aW9uLiBZb3UgY2FuIGRvIHRoZSBmb2xsb3dpbmcgdHJpY2s6DQotIFN0YXJ0IHdpdGggdGhlIGFwcHJvYWNoIGRlc2NyaWJlZCBpbiB0aGUgcHJldmlvdXMgc2VjdGlvbiBhbmQgY2hlY2sgdGhhdCBldmVyeXRoaW5nIGdvZXMgZmluZTsgYXQgdGhpcyBtb21lbnQgd2Ugc2hvdWxkIGhhdmUgYSB2YWxpZCBOWExvZyBjb25maWd1cmF0aW9uIGZpbGUNCi0gTW92ZSB0aGUgY29uZmlndXJhdGlvbiBmaWxlIGluIGFub3RoZXIgZGlyZWN0b3J5IGFuZCBjaGFuZ2UgdGhlIGNvbmZpZ3VyYXRpb25fcGF0aCB2YWx1ZSB0byB0aGF0IGZpbGUNCi0gUmVtb3ZlIGFsbCB2YWxpZCB0YWdzIGFuZCBhZGQgYSBkdW1teSBvbmUgdGhhdCBkb2VzIG5vdCBleGlzdHMgaW4gc2VydmVyIHNpZGUgY29uZmlndXJhdGlvbjsgZG8gbm90IHJlbW92ZSB0aGUgdGFncyBlbnRyeSBvciBsZWF2ZSB0aGUgbGlzdCBlbXB0eSBzaW5jZSB0aGUgY29sbGVjdG9yIHdpbGwgbm90IHN0YXJ0IGFuZCBubyBlcnJvciBtZXNzYWdlIHdpbGwgYmUgbG9nZ2VkDQotIFJlc3RhcnQgdGhlIGNvbGxlY3Rvci1zaWRlY2FyIHNlcnZpY2UNCi0gSWYgZXZlcnl0aGluZyB3b3JrcyBhcyBleHBlY3RlZCwgcmVhZCBjYXJlZnVsbHkgdGhlIE5YTG9nIG1hbnVhbCBhbmQgZG8geW91ciBvd24gZW5oYW5jZW1lbnRzIHRvIHRoZSBjb25maWd1cmF0aW9uDQotIFJlc3RhcnQgdGhlIGNvbGxlY3Rvci1zaWRlY2FyIHNlcnZpY2UNCg0KSSBhcHBsaWVkIHRoZSBhcHByb2FjaCBiZWNhdXNlIEkgbmVlZGVkIGFuIGFkaXRpb25hbCBtZXNzYWdlIGZpZWxkIChgRXhlYyAkc291cmNlX2lkID0gJ1dpbGRGbHkxMCc7YCkgc291cmNlX2lkIGZvciBvbmUgb2YgdGhlIGlucHV0cywgbGlrZSBzaG93biBiZWxvdy4NCmBgYA0KPElucHV0IDU3NmQxZDNkNjU5ZmIxMDdmNzIyZGIzOD4NCglNb2R1bGUgaW1fZmlsZQ0KCUZpbGUgIkM6XEphdmFcd2lsZGZseS0xMC4wLjAuRmluYWxcc3RhbmRhbG9uZVxsb2dcc2VydmVyLmxvZyINCglQb2xsSW50ZXJ2YWwgMQ0KCVNhdmVQb3MJVHJ1ZQ0KCVJlYWRGcm9tTGFzdCBUcnVlDQoJUmVjdXJzaXZlIEZhbHNlDQoJUmVuYW1lQ2hlY2sgRmFsc2UNCglFeGVjICRGaWxlTmFtZSA9IGZpbGVfbmFtZSgpOyAjIFNlbmQgZmlsZSBuYW1lIHdpdGggZWFjaCBtZXNzYWdlDQoJRXhlYyAkc291cmNlX2lkID0gJ1dpbGRGbHkxMCc7DQoJSW5wdXRUeXBlIDU3NmQxZDNkNjU5ZmIxMDdmNzIyZGIzOC1tdWx0aWxpbmUNCjwvSW5wdXQ+DQoNCmBgYA0KDQpUaGUgZnVsbCB5YW1sIGNvbmZpZ3VyYXRpb24gZmlsZSBjb250ZW50Og0KYGBgDQpzZXJ2ZXJfdXJsOiBodHRwOi8vZ3JheWxvZy5teWNvbXBhbnkubG9jYWw6MTI5MDANCnRsc19za2lwX3ZlcmlmeTogZmFsc2UNCm5vZGVfaWQ6IGdyYXlsb2ctY29sbGVjdG9yLXNpZGVjYXItZGV2c3J2DQpjb2xsZWN0b3JfaWQ6IGZpbGU6QzpcUHJvZ3JhbSBGaWxlcyAoeDg2KVxncmF5bG9nXGNvbGxlY3Rvci1zaWRlY2FyXGNvbGxlY3Rvci1pZA0KdGFnczoNCiAgICAtIHVuZGVmaW5lZA0KbG9nX3BhdGg6IEM6XFByb2dyYW0gRmlsZXMgKHg4NilcZ3JheWxvZ1xjb2xsZWN0b3Itc2lkZWNhcg0KdXBkYXRlX2ludGVydmFsOiAxMA0KYmFja2VuZHM6DQogICAgLSBuYW1lOiBueGxvZw0KICAgICAgZW5hYmxlZDogdHJ1ZQ0KICAgICAgYmluYXJ5X3BhdGg6IEM6XFByb2dyYW0gRmlsZXMgKHg4Nilcbnhsb2dcbnhsb2cuZXhlDQojICAgICAgY29uZmlndXJhdGlvbl9wYXRoOiBDOlxQcm9ncmFtIEZpbGVzICh4ODYpXGdyYXlsb2dcY29sbGVjdG9yLXNpZGVjYXJcZ2VuZXJhdGVkXG54bG9nLmNvbmYNCiAgICAgIGNvbmZpZ3VyYXRpb25fcGF0aDogQzpcUHJvZ3JhbSBGaWxlcyAoeDg2KVxncmF5bG9nXGNvbGxlY3Rvci1zaWRlY2FyXGNvbmZcbnhsb2cuY29uZg0KYGBgDQoNCkFzIGFuIGFsdGVybmF0ZSBhcHByb2FjaCAoSSBoYXZlbid0IHRlc3RlZCBidXQgaXQgc2hvdWxkIHdvcmspIGlzIHRvIGdvIGZvciB0aGUgZmlyc3Qgb3B0aW9uIChzZXJ2ZXIgc2lkZSBjb250cm9sbGVkIGNvbmZpZ3VyYXRpb24pIGJ1dCBpbnN0ZWFkIG9mIGRlZmluaW5nIE91dHB1dHMvSW5wdXRzIGp1c3QgcGxhY2UgYWxsIG54bG9nLmNvbmYgY29udGVudCBpbiB0aGUgZGVmYXVsdCBueGxvZyBzbmlwcGV0LCBpbiB0aGUgV2luZG93cyBzZWN0aW9uLiBCZSBzdXJlIHlvdSBoYXZlIGluIHN5bmMgdGhlIGNvbGxlY3RvciBzaWRlIGNsaWVudF9pZCB3aXRoIHRoZSBvbmUgaW4gdGhlIG54bG9nLmNvbmYgY29udGVudCwgT3V0cHV0IHNlY3Rpb24gKHNlZSBgRXhlYyAkZ2wyX3NvdXJjZV9jb2xsZWN0b3IgPSAnMDMyNGJkNzktYjlhOS00NThlLTk5NzctZDBlZWNiZDJkMzQ3JztgKToNCg0KYGBgDQo8T3V0cHV0IDU3NmNmNDI2NjU5ZmIxMDdmNzIyYjAxMT4NCglNb2R1bGUgb21fdGNwDQoJSG9zdCBncmF5bG9nLm15Y29tcGFueS5sb2NhbA0KCVBvcnQgMTIyMDENCglPdXRwdXRUeXBlICBHRUxGX1RDUA0KCUV4ZWMgJHNob3J0X21lc3NhZ2UgPSAkcmF3X2V2ZW50OyAjIEF2b2lkcyB0cnVuY2F0aW9uIG9mIHRoZSBzaG9ydF9tZXNzYWdlIGZpZWxkLg0KCUV4ZWMgJGdsMl9zb3VyY2VfY29sbGVjdG9yID0gJzAzMjRiZDc5LWI5YTktNDU4ZS05OTc3LWQwZWVjYmQyZDM0Nyc7DQoJRXhlYyAkSG9zdG5hbWUgPSBob3N0bmFtZV9mcWRuKCk7DQo8L091dHB1dD4NCmBgYA0KDQojIyMgRHVwbGljYXRlIGNvbmZpZ3VyYXRpb24gb24gYW5vdGhlciBzZXJ2ZXINCg0KT25jZSB3ZSBoYXZlIGEgY29sbGVjdG9yIGNvbmZpZ3VyZWQgd2UgY2FuIGR1cGxpY2F0ZSBlYXNpbHkgdGhlIGluc3RhbGxhdGlvbiBvbiBhbm90aGVyIG9uZToNCg0KDQotIFBlcmZvcm0gdGhlIE5YTG9nIGluc3RhbGxhdGlvbiBzdGVwcw0KLSBJbnN0YWxsIHRoZSBjb2xsZWN0b3Itc2lkZWNhciAoanVzdCBydW4gdGhlIHNldHVwKQ0KLSBDb3B5IHlhbWwgYW5kIG54bG9nLmNvbmYgZmlsZXMgb24gdGhlIHRhcmdldCBzZXJ2ZXINCi0gQ2hhbmdlIHRoZSB5YW1sIHRvIHJlZmxlY3QgdGhlIHJpZ2h0IG5vZGVfaWQNCi0gQ2hhbmdlIHRoZSBjb250ZW50IG9mIHRoZSBjb2xsZWN0b3ItaWQgd2l0aCBzb21lIHVuaXF1ZSBzdHJpbmcgKGEgR1VJRD8pDQotIENoYW5nZSB0aGUgbnhsb2cuY29uZiBPdXRwdXQgdGFnIHRvIHJlZmxlY3QgdGhlIGNvbGxlY3Rvci1pZCB2YWx1ZSAg

Tuesday, May 31, 2016

Cracking Passwords

The problem

We have to face the following situation: we want to migrate our customers from one system (legacy) to another. Customer data (emails,, address, login) are the easy part. What about passwords; wouldn't be nice that the users login into the new system with their actual passwords?

Technological we weren't lucky:

the old system stores passwords as crypt(3) over MD5 (algorithm ID 1); see https://quickhash.com/crypt3-md5-online if you want generate some test data.
the new system accepts SHA-256

so we can't just copy also the hashed password

Remark: by no means assertions in this blog should be considered the final truth: while facts and samples were tested I might missed some options that might further enhance your experience.

The solution

As I hope you understand this is just used for documenting some cracking password tools not a recommendation for a best practice: imagine what customers would thing about your ethics if realizing what you have done...

So, coming back to the topic: try some cracking tools, do a reverse engineering of passwords and then compute the target SHA-256 hash.

Problem solved! Well, not exactly. Having a technical solution is not always equivalent with having the problem actually solved, as we will discover soon.

Tools investigated:

john the ripper (http://www.openwall.com/john/)
hashcat (https://hashcat.net/wiki/)

John

Basic usage is straight forward:

Export hashed password in accepted text format: <username>:<crypt(3)> per line; something like florin:$1$florin$1Y2H3TC.KoSWWN7v9hUZ0.
run john: john passwords.txt

Several variations

Single mode

Run just rules from the configuration file section [List.Rules:Single] (see john.ini)

john --single passwords.txt

Incremental

Based on the Incremental:XXX section definition john will try (brute force) all combinations. See below a customized MyAlpha section:

[Incremental:MyAlpha]
File = $JOHN/alpha.chr
MinLen = 1
MaxLen = 6
CharCount = 26

rem john -i=MyAlpha ../Passwords.txt

Remarks

The good news

Not so good for customers actually: the single mode will be able to quick (in a matter of seconds for the 1000 samples I used) solve 10% of the users by using common patterns applied to their user names. This might not be valid for other samples

The bad news

john is single threaded, so I had to split the passwords.txt into four files and run it in parallel. After several days just some additional tens of passwords were discovered, (1, 2 or 3 letters/digits) so I cad say good by to the rest

Hashcat

It comes into two flavours:

CPU based (https://hashcat.net/hashcat/)
GPU based (https://hashcat.net/oclhashcat/)

If you are lucky enough to have one of the two supported GPU' (AMD or Nvidia) go for the second (several magnitude times faster in my samples!).

The usage looked more complicated than for john (might worth to explore dipper); what I used sort of extensively was the mask attack (https://hashcat.net/wiki/doku.php?id=mask_attack).

Password text file should contain (per line) just the ctrypto(3) part, no user names

Samples (should be the same for CPU based hashcat (hashcat-cli64.exe)):

cudaHashcat64.exe --hash-type=500 -a 3 CryptHashes.txt ?l?l?l?l?l
cudaHashcat64.exe --hash-type=500 -a 3 CryptHashes.txt ?l?l?l?d
cudaHashcat64.exe --hash-type=500 -a 3 CryptHashes.txt ?l?l?l?l?d
cudaHashcat64.exe --hash-type=500 -a 3 CryptHashes.txt ?l?l?l?l?l?d
cudaHashcat64.exe --hash-type=500 -a 3 CryptHashes.txt ?l?l?l?l?l?l?d
cudaHashcat64.exe --increment --increment-min=4 --hash-type=500 -a 3 CryptHashes.txt -1 ?l?d ?1?1?1?1?1?1

You can use up to 4 custom char set definitions, something like:

cudaHashcat64.exe --hash-type=500 -a 3 CryptHashes.txt -1 ?u -2 ?l -3 ?d ?1?2?2?2?3?3

This will mean try all combinations that start with an upper letter, contains 3 lower case letters and ends up into two figures.

Best guess: there is a tool that based on mask can generate word files on which you could run hashcat by applying rules. Just putting (once again, as best guess) together what I remember from reading chaotically the documentation.

The Good News

If passwords are short enough (5~6 in length) and you will be able to guess the right mask the GPU based version will might do the job (like 20 hours for ?l?l?l?l?l mask). CPU load will be minimal.

The Bad News

If the users were clever enough to use longer passwords, combination of Upper/lower/digits/special character spread all over the string you are stuck in months/years of waiting

Common Remarks

Found passwords are stored in john.pot, respectively *hashcat.pot Both tools are re-start-able:

cold restart: just restart the process and found passwords are skipped (but the process is started from the beginning on a smaller list of hashes)
warm restart, based on a saved session which will actually restart were from it was interrupted:
- tested tor cudaHashcat: provide a session name at first run and use that session name for restore

cudaHashcat64.exe --hash-type=500 -a 3 --session=Session CryptHashes10.txt  -1 abcdefghijlmnoprstuv ?1?1?1?1?1?1
...
cudaHashcat64.exe --session=Session --restore

Be aware: Do not quit immediately after restarting in restore mode; wait till the current restore point changes from zero to something near the actual progress value otherwise you will loose all work done so far. This is normal behaviour as stated on some answers on forum

IyBUaGUgcHJvYmxlbQ0KDQpXZSBoYXZlIHRvIGZhY2UgdGhlIGZvbGxvd2luZyBzaXR1YXRpb246IHdlIHdhbnQgdG8gbWlncmF0ZSBvdXIgY3VzdG9tZXJzIGZyb20gb25lIHN5c3RlbSAobGVnYWN5KSB0byBhbm90aGVyLiBDdXN0b21lciBkYXRhIChlbWFpbHMsLCBhZGRyZXNzLCBsb2dpbikgYXJlIHRoZSBlYXN5IHBhcnQuIFdoYXQgYWJvdXQgcGFzc3dvcmRzOyB3b3VsZG4ndCBiZSBuaWNlIHRoYXQgdGhlIHVzZXJzIGxvZ2luIGludG8gdGhlIG5ldyBzeXN0ZW0gd2l0aCB0aGVpciBhY3R1YWwgcGFzc3dvcmRzPw0KDQpUZWNobm9sb2dpY2FsIHdlIHdlcmVuJ3QgbHVja3k6DQotIHRoZSBvbGQgc3lzdGVtIHN0b3JlcyBwYXNzd29yZHMgYXMgY3J5cHQoMykgb3ZlciBNRDUgKGFsZ29yaXRobSBJRCAxKTsgc2VlIFtodHRwczovL3F1aWNraGFzaC5jb20vY3J5cHQzLW1kNS1vbmxpbmVdKGh0dHBzOi8vcXVpY2toYXNoLmNvbS9jcnlwdDMtbWQ1LW9ubGluZSkgaWYgeW91IHdhbnQgZ2VuZXJhdGUgc29tZSB0ZXN0IGRhdGEuDQotIHRoZSBuZXcgc3lzdGVtIGFjY2VwdHMgU0hBLTI1Ng0KDQpzbyB3ZSBjYW4ndCBqdXN0IGNvcHkgYWxzbyB0aGUgaGFzaGVkIHBhc3N3b3JkDQoNCioqUmVtYXJrKio6IGJ5IG5vIG1lYW5zIGFzc2VydGlvbnMgaW4gdGhpcyBibG9nIHNob3VsZCBiZSBjb25zaWRlcmVkIHRoZSBmaW5hbCB0cnV0aDogd2hpbGUgZmFjdHMgYW5kIHNhbXBsZXMgd2VyZSB0ZXN0ZWQgSSBtaWdodCBtaXNzZWQgc29tZSBvcHRpb25zIHRoYXQgbWlnaHQgZnVydGhlciBlbmhhbmNlIHlvdXIgZXhwZXJpZW5jZS4NCg0KIyBUaGUgc29sdXRpb24NCg0KKkFzIEkgaG9wZSB5b3UgdW5kZXJzdGFuZCB0aGlzIGlzIGp1c3QgdXNlZCBmb3IgZG9jdW1lbnRpbmcgc29tZSBjcmFja2luZyBwYXNzd29yZCB0b29scyBub3QgYSByZWNvbW1lbmRhdGlvbiBmb3IgYSBiZXN0IHByYWN0aWNlOiBpbWFnaW5lIHdoYXQgY3VzdG9tZXJzIHdvdWxkIHRoaW5nIGFib3V0IHlvdXIgZXRoaWNzIGlmIHJlYWxpemluZyB3aGF0IHlvdSBoYXZlIGRvbmUuLi4qDQoNClNvLCBjb21pbmcgYmFjayB0byB0aGUgdG9waWM6IHRyeSBzb21lIGNyYWNraW5nIHRvb2xzLCBkbyBhIHJldmVyc2UgZW5naW5lZXJpbmcgb2YgcGFzc3dvcmRzIGFuZCB0aGVuIGNvbXB1dGUgdGhlIHRhcmdldCBTSEEtMjU2IGhhc2guDQoNClByb2JsZW0gc29sdmVkISBXZWxsLCBub3QgZXhhY3RseS4gSGF2aW5nIGEgdGVjaG5pY2FsIHNvbHV0aW9uIGlzIG5vdCBhbHdheXMgZXF1aXZhbGVudCB3aXRoIGhhdmluZyB0aGUgcHJvYmxlbSBhY3R1YWxseSBzb2x2ZWQsIGFzIHdlIHdpbGwgZGlzY292ZXIgc29vbi4NCg0KVG9vbHMgaW52ZXN0aWdhdGVkOg0KLSBqb2huIHRoZSByaXBwZXIgKFtodHRwOi8vd3d3Lm9wZW53YWxsLmNvbS9qb2huL10oaHR0cDovL3d3dy5vcGVud2FsbC5jb20vam9obi8pKQ0KLSBoYXNoY2F0IChbaHR0cHM6Ly9oYXNoY2F0Lm5ldC93aWtpL10oaHR0cHM6Ly9oYXNoY2F0Lm5ldC93aWtpLykpDQoNCiMjIEpvaG4NCg0KQmFzaWMgdXNhZ2UgaXMgc3RyYWlnaHQgZm9yd2FyZDoNCi0gRXhwb3J0IGhhc2hlZCBwYXNzd29yZCBpbiBhY2NlcHRlZCB0ZXh0IGZvcm1hdDogYDx1c2VybmFtZT46PGNyeXB0KDMpPmAgcGVyIGxpbmU7IHNvbWV0aGluZyBsaWtlIGBmbG9yaW46JDEkZmxvcmluJDFZMkgzVEMuS29TV1dON3Y5aFVaMC5gDQotIHJ1biBqb2huOiBgam9obiBwYXNzd29yZHMudHh0YA0KDQojIyMgU2V2ZXJhbCB2YXJpYXRpb25zDQoNCiMjIyMgU2luZ2xlIG1vZGUNCg0KUnVuIGp1c3QgcnVsZXMgZnJvbSB0aGUgY29uZmlndXJhdGlvbiBmaWxlIHNlY3Rpb24gW0xpc3QuUnVsZXM6U2luZ2xlXSAoc2VlIGpvaG4uaW5pKQ0KDQpgam9obiAtLXNpbmdsZSBwYXNzd29yZHMudHh0YA0KDQojIyMjIEluY3JlbWVudGFsDQoNCkJhc2VkIG9uIHRoZSBgSW5jcmVtZW50YWw6WFhYYCBzZWN0aW9uIGRlZmluaXRpb24gam9obiB3aWxsIHRyeSAoYnJ1dGUgZm9yY2UpIGFsbCBjb21iaW5hdGlvbnMuIFNlZSBiZWxvdyBhIGN1c3RvbWl6ZWQgTXlBbHBoYSBzZWN0aW9uOg0KDQpgYGANCltJbmNyZW1lbnRhbDpNeUFscGhhXQ0KRmlsZSA9ICRKT0hOL2FscGhhLmNocg0KTWluTGVuID0gMQ0KTWF4TGVuID0gNg0KQ2hhckNvdW50ID0gMjYNCmBgYA0KYHJlbSBqb2huIC1pPU15QWxwaGEgLi4vUGFzc3dvcmRzLnR4dGANCg0KIyMjIFJlbWFya3MNCg0KIyMjIyBUaGUgZ29vZCBuZXdzDQoNCk5vdCBzbyBnb29kIGZvciBjdXN0b21lcnMgYWN0dWFsbHk6IHRoZSBzaW5nbGUgbW9kZSB3aWxsIGJlIGFibGUgdG8gcXVpY2sgKGluIGEgbWF0dGVyIG9mIHNlY29uZHMgZm9yIHRoZSAxMDAwIHNhbXBsZXMgSSB1c2VkKSBzb2x2ZSAxMCUgb2YgdGhlIHVzZXJzIGJ5IHVzaW5nIGNvbW1vbiBwYXR0ZXJucyBhcHBsaWVkIHRvIHRoZWlyIHVzZXIgbmFtZXMuIFRoaXMgbWlnaHQgbm90IGJlIHZhbGlkIGZvciBvdGhlciBzYW1wbGVzDQoNCiMjIyMgVGhlIGJhZCBuZXdzDQoNCioqam9obioqIGlzIHNpbmdsZSB0aHJlYWRlZCwgc28gSSBoYWQgdG8gc3BsaXQgdGhlIHBhc3N3b3Jkcy50eHQgaW50byBmb3VyIGZpbGVzIGFuZCBydW4gaXQgaW4gcGFyYWxsZWwuIEFmdGVyIHNldmVyYWwgZGF5cyBqdXN0IHNvbWUgYWRkaXRpb25hbCB0ZW5zIG9mIHBhc3N3b3JkcyB3ZXJlIGRpc2NvdmVyZWQsICgxLCAyIG9yIDMgbGV0dGVycy9kaWdpdHMpIHNvIEkgY2FkIHNheSBnb29kIGJ5IHRvIHRoZSByZXN0DQoNCiMjIEhhc2hjYXQNCg0KSXQgY29tZXMgaW50byB0d28gZmxhdm91cnM6DQotIENQVSBiYXNlZCAoW2h0dHBzOi8vaGFzaGNhdC5uZXQvaGFzaGNhdC9dKGh0dHBzOi8vaGFzaGNhdC5uZXQvaGFzaGNhdC8pKQ0KLSBHUFUgYmFzZWQgKFtodHRwczovL2hhc2hjYXQubmV0L29jbGhhc2hjYXQvXShodHRwczovL2hhc2hjYXQubmV0L29jbGhhc2hjYXQvKSkNCg0KSWYgeW91IGFyZSBsdWNreSBlbm91Z2ggdG8gaGF2ZSBvbmUgb2YgdGhlIHR3byBzdXBwb3J0ZWQgR1BVJyAoQU1EIG9yIE52aWRpYSkgZ28gZm9yIHRoZSBzZWNvbmQgKHNldmVyYWwgbWFnbml0dWRlIHRpbWVzIGZhc3RlciBpbiBteSBzYW1wbGVzISkuDQoNClRoZSB1c2FnZSBsb29rZWQgbW9yZSBjb21wbGljYXRlZCB0aGFuIGZvciBqb2huIChtaWdodCB3b3J0aCB0byBleHBsb3JlIGRpcHBlcik7IHdoYXQgSSB1c2VkIHNvcnQgb2YgZXh0ZW5zaXZlbHkgd2FzIHRoZSBgbWFzayBhdHRhY2tgIChbaHR0cHM6Ly9oYXNoY2F0Lm5ldC93aWtpL2Rva3UucGhwP2lkPW1hc2tfYXR0YWNrXShodHRwczovL2hhc2hjYXQubmV0L3dpa2kvZG9rdS5waHA/aWQ9bWFza19hdHRhY2spKS4NCg0KUGFzc3dvcmQgdGV4dCBmaWxlIHNob3VsZCBjb250YWluIChwZXIgbGluZSkganVzdCB0aGUgY3RyeXB0bygzKSBwYXJ0LCBubyB1c2VyIG5hbWVzDQoNClNhbXBsZXMgKHNob3VsZCBiZSB0aGUgc2FtZSBmb3IgQ1BVIGJhc2VkIGhhc2hjYXQgKGBoYXNoY2F0LWNsaTY0LmV4ZWApKToNCmBgYA0KY3VkYUhhc2hjYXQ2NC5leGUgLS1oYXNoLXR5cGU9NTAwIC1hIDMgQ3J5cHRIYXNoZXMudHh0ID9sP2w/bD9sP2wNCmN1ZGFIYXNoY2F0NjQuZXhlIC0taGFzaC10eXBlPTUwMCAtYSAzIENyeXB0SGFzaGVzLnR4dCA/bD9sP2w/ZA0KY3VkYUhhc2hjYXQ2NC5leGUgLS1oYXNoLXR5cGU9NTAwIC1hIDMgQ3J5cHRIYXNoZXMudHh0ID9sP2w/bD9sP2QNCmN1ZGFIYXNoY2F0NjQuZXhlIC0taGFzaC10eXBlPTUwMCAtYSAzIENyeXB0SGFzaGVzLnR4dCA/bD9sP2w/bD9sP2QNCmN1ZGFIYXNoY2F0NjQuZXhlIC0taGFzaC10eXBlPTUwMCAtYSAzIENyeXB0SGFzaGVzLnR4dCA/bD9sP2w/bD9sP2w/ZA0KY3VkYUhhc2hjYXQ2NC5leGUgLS1pbmNyZW1lbnQgLS1pbmNyZW1lbnQtbWluPTQgLS1oYXNoLXR5cGU9NTAwIC1hIDMgQ3J5cHRIYXNoZXMudHh0IC0xID9sP2QgPzE/MT8xPzE/MT8xDQoNCmBgYA0KWW91IGNhbiB1c2UgdXAgdG8gNCBjdXN0b20gY2hhciBzZXQgZGVmaW5pdGlvbnMsIHNvbWV0aGluZyBsaWtlOg0KYGBgDQpjdWRhSGFzaGNhdDY0LmV4ZSAtLWhhc2gtdHlwZT01MDAgLWEgMyBDcnlwdEhhc2hlcy50eHQgLTEgP3UgLTIgP2wgLTMgP2QgPzE/Mj8yPzI/Mz8zDQpgYGANClRoaXMgd2lsbCBtZWFuIHRyeSBhbGwgY29tYmluYXRpb25zIHRoYXQgc3RhcnQgd2l0aCBhbiB1cHBlciBsZXR0ZXIsIGNvbnRhaW5zIDMgbG93ZXIgY2FzZSBsZXR0ZXJzIGFuZCBlbmRzIHVwIGludG8gdHdvIGZpZ3VyZXMuDQoNCg0KQmVzdCBndWVzczogdGhlcmUgaXMgYSB0b29sIHRoYXQgYmFzZWQgb24gbWFzayBjYW4gZ2VuZXJhdGUgd29yZCBmaWxlcyBvbiB3aGljaCB5b3UgY291bGQgcnVuIGhhc2hjYXQgYnkgYXBwbHlpbmcgcnVsZXMuIEp1c3QgcHV0dGluZyAob25jZSBhZ2FpbiwgYXMgYmVzdCBndWVzcykgdG9nZXRoZXIgd2hhdCBJIHJlbWVtYmVyIGZyb20gcmVhZGluZyBjaGFvdGljYWxseSB0aGUgZG9jdW1lbnRhdGlvbi4NCg0KIyMjIFRoZSBHb29kIE5ld3MNCg0KSWYgcGFzc3dvcmRzIGFyZSBzaG9ydCBlbm91Z2ggKDV+NiBpbiBsZW5ndGgpIGFuZCB5b3Ugd2lsbCBiZSBhYmxlIHRvIGd1ZXNzIHRoZSByaWdodCBtYXNrIHRoZSBHUFUgYmFzZWQgdmVyc2lvbiB3aWxsIG1pZ2h0IGRvIHRoZSBqb2IgKGxpa2UgMjAgaG91cnMgZm9yID9sP2w/bD9sP2wgbWFzaykuIENQVSBsb2FkIHdpbGwgYmUgbWluaW1hbC4NCg0KDQojIyMgVGhlIEJhZCBOZXdzDQpJZiB0aGUgdXNlcnMgd2VyZSBjbGV2ZXIgZW5vdWdoIHRvIHVzZSBsb25nZXIgcGFzc3dvcmRzLCBjb21iaW5hdGlvbiBvZiBVcHBlci9sb3dlci9kaWdpdHMvc3BlY2lhbCBjaGFyYWN0ZXIgc3ByZWFkIGFsbCBvdmVyIHRoZSBzdHJpbmcgeW91IGFyZSBzdHVjayBpbiBtb250aHMveWVhcnMgb2Ygd2FpdGluZw0KDQojIENvbW1vbiBSZW1hcmtzDQoNCkZvdW5kIHBhc3N3b3JkcyBhcmUgc3RvcmVkIGluIGpvaG4ucG90LCByZXNwZWN0aXZlbHkgKmhhc2hjYXQucG90DQpCb3RoIHRvb2xzIGFyZSByZS1zdGFydC1hYmxlOg0KDQotIGNvbGQgcmVzdGFydDoganVzdCByZXN0YXJ0IHRoZSBwcm9jZXNzIGFuZCBmb3VuZCBwYXNzd29yZHMgYXJlIHNraXBwZWQgKGJ1dCB0aGUgcHJvY2VzcyBpcyBzdGFydGVkIGZyb20gdGhlIGJlZ2lubmluZyBvbiBhIHNtYWxsZXIgbGlzdCBvZiBoYXNoZXMpDQotIHdhcm0gcmVzdGFydCwgYmFzZWQgb24gYSBzYXZlZCBzZXNzaW9uIHdoaWNoIHdpbGwgYWN0dWFsbHkgcmVzdGFydCB3ZXJlIGZyb20gaXQgd2FzIGludGVycnVwdGVkOg0KCS0gdGVzdGVkIHRvciBjdWRhSGFzaGNhdDogcHJvdmlkZSBhIHNlc3Npb24gbmFtZSBhdCBmaXJzdCBydW4gYW5kIHVzZSB0aGF0IHNlc3Npb24gbmFtZSBmb3IgcmVzdG9yZQ0KYGBgDQpjdWRhSGFzaGNhdDY0LmV4ZSAtLWhhc2gtdHlwZT01MDAgLWEgMyAtLXNlc3Npb249U2Vzc2lvbiBDcnlwdEhhc2hlczEwLnR4dCAgLTEgYWJjZGVmZ2hpamxtbm9wcnN0dXYgPzE/MT8xPzE/MT8xDQouLi4NCmN1ZGFIYXNoY2F0NjQuZXhlIC0tc2Vzc2lvbj1TZXNzaW9uIC0tcmVzdG9yZQ0KYGBgDQoqKkJlIGF3YXJlOioqIERvIG5vdCBxdWl0IGltbWVkaWF0ZWx5IGFmdGVyIHJlc3RhcnRpbmcgaW4gcmVzdG9yZSBtb2RlOyB3YWl0IHRpbGwgdGhlIGN1cnJlbnQgcmVzdG9yZSBwb2ludCBjaGFuZ2VzIGZyb20gemVybyB0byBzb21ldGhpbmcgbmVhciB0aGUgYWN0dWFsIHByb2dyZXNzIHZhbHVlIG90aGVyd2lzZSB5b3Ugd2lsbCBsb29zZSBhbGwgd29yayBkb25lIHNvIGZhci4gVGhpcyBpcyBub3JtYWwgYmVoYXZpb3VyIGFzIHN0YXRlZCBvbiBzb21lIGFuc3dlcnMgb24gZm9ydW0s

Monday, May 16, 2016

MySQL Linked Server from SQL Server 2014

It might work from other SQL Server versions...

Install MySQL ODBC connector

Download and install latest x64 MySQL ODBC connector; version 5.3.6 at the moment of this post.
Create an x64 ODBC System DSN; suppose we named it MySQLDS

Create the Linked Server

Add a Linked Server
- Name: at your will, suppose MySQLLS
- Provider: Microsoft OLEDB Provider for ODBC Drivers
- Product Name: MySQL
- Data Source: MySQLDS
- Provider String:ODBC;DSN=MySQLDS
- Location: (empty)
- Catalog:<name of desired catalog>
On Security tab:
- Check Be made using this security context:
  - root (not really very secure)
  - <password>

Change Provider Options (Linked Servers->Providesr->MSDASQL)

Check Level zero only; this will allow sentences like SELECT * FROM MySQLLS...Table

Saturday, April 16, 2016

javax.transaction.Transactional

Suppose we have a managed CDI bean exposing to JSF-EL two methods:

public void methodA(){
    //do some additional stuff
    methodB();
}
@Transactional
public void methodB(){
    //do actual work by several EJB calls
}

At a first glance you could think:

If we call methodA(from a JSF view) we would get a transaction started and all work will be done inside this transaction
If we call methodA we will get the same result for whatever methodA is doing

Well, while the first assumption works as stated the second is, surprise, wrong. Each individual call to the mentioned EJB will trigger it's own transaction.

This behaviour is due to the following not obvious at first glance fact: JSF will call CDI methods through a proxy that wraps the actual instance and fires interceptors; in our second case interceptors have been already fired before methodA was actually called and no other interceptors (that would trigger transaction management) will be called due to the fact that instance not proxy methodB will be called.

Wednesday, April 13, 2016

SQL Server AlwaysOn

Installing/configuring SQL Server 2014 AlwaysOn Availability Group

The basic desired scenario was a configuration with a primary database (writeable) and a secondary (readable) replica, targeting reports (no actual clustering/fail-over).

The main goal of this post is to point out some prerequisites/main steps not always obvious/present in internet posts/youtube videos.

Installing WSFC (Windows Server fail-over Cluster)

Run the configuration steps as an user with rights to create AD objects. Being a local admin will not suffice. Install first, on both servers (Win 2012 Standard in my case) Failover Cluster feature.

From one of the servers (I run it from main server, I suppose there is no difference) run Create Cluster from Failover Cluster Management.

Add both servers, run validation (for sure you will get some warnings about storage since we do not have any common storage, maybe about the network, as a single point of failure and some warnings about not having the same updates applied on both server; ignore them).

Create the cluster; since in my infrastructure I had DHCP enabled no cluster IP configuration was required.

Configuring AlwaysOn

The operations described below were done with an domain account, member of local Adminsitrators Windows Group and member of sysadmin SQL Server group.

On each server:

Open for inbound access TCP port 5022
Have the same directory structure where databases file support will be installed
Have installed SQL Server 2014, Enterprise or Developer
From SQL Server configuration->SQL Server Configuration Manager->SQL Server Service->Properties->AlwaysOn High Availability tab->Enable AlwaysOn Availability Groups
Restart SQL Server service

On main server (my choice, can be anywhere in the LAN):

create a share \\Server\MyShare with everyone full access (this is required for initial synchronization since we want to leave the default SQL service user assignment (NT Service\MSSQLSERVER)).

Now, on main server open SQL Enterprise Management Studio, go to AlwaysOn High Availability->Availability Groups->New Availability Group Wizard. Choose the database, secondary replica, share for initial synchronization, Endpoint URL, etc. I set (for both servers):

Availability Mode to Synchronous (virtual servers are in the same ESX)
Failover to Manual
Availability to Allow all Connections
Readable Secondary Yes

Backup preferences was set to Primary since I already had a full backup daily/log backup every 15 minutes Maintenance Jobs on primary

When creating the group initial join for secondary replica will fail since (as stated above) we will not change the default user accounts for SQL Service.

Go on each server and create/grant rights for pair computer accounts by running:

 /*
 USE master;
 GO
 SELECT name, state_desc, port from sys.tcp_endpoints where type_desc='DATABASE_MIRRORING'
 GO
*/
 USE master;
 GO
 CREATE LOGIN [DOMAIN\SERVER$] FROM WINDOWS;
 GO
 GRANT CONNECT on ENDPOINT::Hadr_endpoint TO [DOMAIN\SERVER$];
 GO

Go back to SQL Enterprise management studio and manually join the secondary database.

Now we should have a working AlwaysOn Availability Group.

A final challenge now: all database objects are automatically synchronised (DML as well as DDL). What about logins?

We want to add a secondarylogin that has read only access to the database, mainly for accessing it on secondary replica. By adding the login on primary server the AlwaysOn technology will synchronously create the login in secondary database but...no surprise...will not create the SQL Server wide login; so we will get a 'traditional' orphan login. Since the database is read only we can not use the standard fix for orphan users. So, read https://support.microsoft.com/en-us/kb/918992 on how to create clone a SQL Server login from one instance to another (altogether with SID and password hash).

IyBJbnN0YWxsaW5nL2NvbmZpZ3VyaW5nIFNRTCBTZXJ2ZXIgMjAxNCBBbHdheXNPbiBBdmFpbGFiaWxpdHkgR3JvdXANCg0KVGhlIGJhc2ljIGRlc2lyZWQgc2NlbmFyaW8gd2FzIGEgY29uZmlndXJhdGlvbiB3aXRoIGEgcHJpbWFyeSBkYXRhYmFzZSAod3JpdGVhYmxlKSBhbmQgYSBzZWNvbmRhcnkgKHJlYWRhYmxlKSByZXBsaWNhLCB0YXJnZXRpbmcgcmVwb3J0cyAobm8gYWN0dWFsIGNsdXN0ZXJpbmcvZmFpbC1vdmVyKS4gDQoNClRoZSBtYWluIGdvYWwgb2YgdGhpcyBwb3N0IGlzIHRvIHBvaW50IG91dCBzb21lIHByZXJlcXVpc2l0ZXMvbWFpbiBzdGVwcyBub3QgYWx3YXlzIG9idmlvdXMvcHJlc2VudCBpbiBpbnRlcm5ldCBwb3N0cy95b3V0dWJlIHZpZGVvcy4NCg0KIyMgSW5zdGFsbGluZyBXU0ZDIChXaW5kb3dzIFNlcnZlciBmYWlsLW92ZXIgQ2x1c3RlcikNCg0KUnVuIHRoZSBjb25maWd1cmF0aW9uIHN0ZXBzIGFzIGFuIHVzZXIgd2l0aCByaWdodHMgdG8gY3JlYXRlIEFEIG9iamVjdHMuIEJlaW5nIGEgbG9jYWwgYWRtaW4gd2lsbCBub3Qgc3VmZmljZS4NCkluc3RhbGwgZmlyc3QsIG9uIGJvdGggc2VydmVycyAoV2luIDIwMTIgU3RhbmRhcmQgaW4gbXkgY2FzZSkgRmFpbG92ZXIgQ2x1c3RlciBmZWF0dXJlLg0KDQpGcm9tIG9uZSBvZiB0aGUgc2VydmVycyAoSSBydW4gaXQgZnJvbSBtYWluIHNlcnZlciwgSSBzdXBwb3NlIHRoZXJlIGlzIG5vIGRpZmZlcmVuY2UpIHJ1biBDcmVhdGUgQ2x1c3RlciBmcm9tIEZhaWxvdmVyIENsdXN0ZXIgTWFuYWdlbWVudC4NCg0KQWRkIGJvdGggc2VydmVycywgcnVuIHZhbGlkYXRpb24gKGZvciBzdXJlIHlvdSB3aWxsIGdldCBzb21lIHdhcm5pbmdzIGFib3V0IHN0b3JhZ2Ugc2luY2Ugd2UgZG8gbm90IGhhdmUgYW55IGNvbW1vbiBzdG9yYWdlLCBtYXliZSBhYm91dCB0aGUgbmV0d29yaywgYXMgYSBzaW5nbGUgcG9pbnQgb2YgZmFpbHVyZSBhbmQgc29tZSB3YXJuaW5ncyBhYm91dCBub3QgaGF2aW5nIHRoZSBzYW1lIHVwZGF0ZXMgYXBwbGllZCBvbiBib3RoIHNlcnZlcjsgaWdub3JlIHRoZW0pLiANCg0KQ3JlYXRlIHRoZSBjbHVzdGVyOyBzaW5jZSBpbiBteSBpbmZyYXN0cnVjdHVyZSBJIGhhZCBESENQIGVuYWJsZWQgbm8gY2x1c3RlciBJUCBjb25maWd1cmF0aW9uIHdhcyByZXF1aXJlZC4NCg0KIyMgQ29uZmlndXJpbmcgQWx3YXlzT24NCg0KVGhlIG9wZXJhdGlvbnMgZGVzY3JpYmVkIGJlbG93IHdlcmUgZG9uZSB3aXRoIGFuIGRvbWFpbiBhY2NvdW50LCBtZW1iZXIgb2YgbG9jYWwgQWRtaW5zaXRyYXRvcnMgV2luZG93cyBHcm91cCBhbmQgbWVtYmVyIG9mIHN5c2FkbWluIFNRTCBTZXJ2ZXIgZ3JvdXAuDQoNCk9uIGVhY2ggc2VydmVyOg0KLSBPcGVuIGZvciBpbmJvdW5kIGFjY2VzcyBUQ1AgcG9ydCA1MDIyDQotIEhhdmUgdGhlIHNhbWUgZGlyZWN0b3J5IHN0cnVjdHVyZSB3aGVyZSBkYXRhYmFzZXMgZmlsZSBzdXBwb3J0IHdpbGwgYmUgaW5zdGFsbGVkDQotIEhhdmUgaW5zdGFsbGVkIFNRTCBTZXJ2ZXIgMjAxNCwgRW50ZXJwcmlzZSBvciBEZXZlbG9wZXINCi0gRnJvbSBTUUwgU2VydmVyIGNvbmZpZ3VyYXRpb24tPlNRTCBTZXJ2ZXIgQ29uZmlndXJhdGlvbiBNYW5hZ2VyLT5TUUwgU2VydmVyIFNlcnZpY2UtPlByb3BlcnRpZXMtPkFsd2F5c09uIEhpZ2ggQXZhaWxhYmlsaXR5IHRhYi0+RW5hYmxlIEFsd2F5c09uIEF2YWlsYWJpbGl0eSBHcm91cHMNCi0gUmVzdGFydCBTUUwgU2VydmVyIHNlcnZpY2UNCg0KT24gbWFpbiBzZXJ2ZXIgKG15IGNob2ljZSwgY2FuIGJlIGFueXdoZXJlIGluIHRoZSBMQU4pOg0KLSBjcmVhdGUgYSBzaGFyZSBcXFxcU2VydmVyXFxNeVNoYXJlIHdpdGggZXZlcnlvbmUgZnVsbCBhY2Nlc3MgKHRoaXMgaXMgcmVxdWlyZWQgZm9yIGluaXRpYWwgc3luY2hyb25pemF0aW9uIHNpbmNlIHdlIHdhbnQgdG8gbGVhdmUgdGhlIGRlZmF1bHQgU1FMIHNlcnZpY2UgdXNlciBhc3NpZ25tZW50IChOVCBTZXJ2aWNlXE1TU1FMU0VSVkVSKSkuDQoNCk5vdywgb24gbWFpbiBzZXJ2ZXIgb3BlbiBTUUwgRW50ZXJwcmlzZSBNYW5hZ2VtZW50IFN0dWRpbywgZ28gdG8gQWx3YXlzT24gSGlnaCBBdmFpbGFiaWxpdHktPkF2YWlsYWJpbGl0eSBHcm91cHMtPk5ldyBBdmFpbGFiaWxpdHkgR3JvdXAgV2l6YXJkLg0KQ2hvb3NlIHRoZSBkYXRhYmFzZSwgc2Vjb25kYXJ5IHJlcGxpY2EsIHNoYXJlIGZvciBpbml0aWFsIHN5bmNocm9uaXphdGlvbiwgRW5kcG9pbnQgVVJMLCBldGMuIA0KSSBzZXQgKGZvciBib3RoIHNlcnZlcnMpOg0KDQotIEF2YWlsYWJpbGl0eSBNb2RlIHRvIFN5bmNocm9ub3VzICh2aXJ0dWFsIHNlcnZlcnMgYXJlIGluIHRoZSBzYW1lIEVTWCkgDQotIEZhaWxvdmVyIHRvIE1hbnVhbA0KLSBBdmFpbGFiaWxpdHkgdG8gQWxsb3cgYWxsIENvbm5lY3Rpb25zDQotIFJlYWRhYmxlIFNlY29uZGFyeSBZZXMNCg0KQmFja3VwIHByZWZlcmVuY2VzIHdhcyBzZXQgdG8gUHJpbWFyeSBzaW5jZSBJIGFscmVhZHkgaGFkIGEgZnVsbCBiYWNrdXAgZGFpbHkvbG9nIGJhY2t1cCBldmVyeSAxNSBtaW51dGVzIE1haW50ZW5hbmNlIEpvYnMgb24gcHJpbWFyeQ0KDQpXaGVuIGNyZWF0aW5nIHRoZSBncm91cCBpbml0aWFsIGpvaW4gZm9yIHNlY29uZGFyeSByZXBsaWNhIHdpbGwgZmFpbCBzaW5jZSAoYXMgc3RhdGVkIGFib3ZlKSB3ZSB3aWxsIG5vdCBjaGFuZ2UgdGhlIA0KZGVmYXVsdCB1c2VyIGFjY291bnRzIGZvciBTUUwgU2VydmljZS4NCg0KR28gb24gZWFjaCBzZXJ2ZXIgYW5kIGNyZWF0ZS9ncmFudCByaWdodHMgZm9yIHBhaXIgY29tcHV0ZXIgYWNjb3VudHMgYnkgcnVubmluZzoNCg0KYGBgU1FMDQogLyoNCiBVU0UgbWFzdGVyOw0KIEdPDQogU0VMRUNUIG5hbWUsIHN0YXRlX2Rlc2MsIHBvcnQgZnJvbSBzeXMudGNwX2VuZHBvaW50cyB3aGVyZSB0eXBlX2Rlc2M9J0RBVEFCQVNFX01JUlJPUklORycNCiBHTw0KKi8NCiBVU0UgbWFzdGVyOw0KIEdPDQogQ1JFQVRFIExPR0lOIFtET01BSU5cU0VSVkVSJF0gRlJPTSBXSU5ET1dTOw0KIEdPDQogR1JBTlQgQ09OTkVDVCBvbiBFTkRQT0lOVDo6SGFkcl9lbmRwb2ludCBUTyBbRE9NQUlOXFNFUlZFUiRdOw0KIEdPDQpgYGANClJlYWQgbW9yZSBvbiBbaHR0cHM6Ly9ibG9ncy5tc2RuLm1pY3Jvc29mdC5jb20vc3ZhcnVrYWxhLzIwMTQvMDMvMzEvc3FsLWFsd2F5c29uLWZhaWxlZC10by1qb2luLXRoZS1kYXRhYmFzZS10by10aGUtYXZhaWxhYmlsaXR5LWdyb3VwLWVycm9yLTM1MjUwL10oaHR0cHM6Ly9ibG9ncy5tc2RuLm1pY3Jvc29mdC5jb20vc3ZhcnVrYWxhLzIwMTQvMDMvMzEvc3FsLWFsd2F5c29uLWZhaWxlZC10by1qb2luLXRoZS1kYXRhYmFzZS10by10aGUtYXZhaWxhYmlsaXR5LWdyb3VwLWVycm9yLTM1MjUwLykNCg0KR28gYmFjayB0byBTUUwgRW50ZXJwcmlzZSBtYW5hZ2VtZW50IHN0dWRpbyBhbmQgbWFudWFsbHkgam9pbiB0aGUgc2Vjb25kYXJ5IGRhdGFiYXNlLg0KDQpOb3cgd2Ugc2hvdWxkIGhhdmUgYSB3b3JraW5nIEFsd2F5c09uIEF2YWlsYWJpbGl0eSBHcm91cC4NCg0KQSBmaW5hbCBjaGFsbGVuZ2Ugbm93OiBhbGwgZGF0YWJhc2Ugb2JqZWN0cyBhcmUgYXV0b21hdGljYWxseSBzeW5jaHJvbmlzZWQgKERNTCBhcyB3ZWxsIGFzIERETCkuIFdoYXQgYWJvdXQgbG9naW5zPw0KDQpXZSB3YW50IHRvIGFkZCBhIGBzZWNvbmRhcnlsb2dpbmAgdGhhdCBoYXMgcmVhZCBvbmx5IGFjY2VzcyB0byB0aGUgZGF0YWJhc2UsIG1haW5seSBmb3IgYWNjZXNzaW5nIGl0IG9uIHNlY29uZGFyeSByZXBsaWNhLiBCeSBhZGRpbmcgdGhlIGxvZ2luIG9uIHByaW1hcnkgc2VydmVyIHRoZSBBbHdheXNPbiB0ZWNobm9sb2d5IHdpbGwgc3luY2hyb25vdXNseSBjcmVhdGUgdGhlIGxvZ2luIGluIHNlY29uZGFyeSBkYXRhYmFzZSBidXQuLi5ubyBzdXJwcmlzZS4uLndpbGwgbm90IGNyZWF0ZSB0aGUgU1FMIFNlcnZlciB3aWRlIGxvZ2luOyBzbyB3ZSB3aWxsIGdldCBhICd0cmFkaXRpb25hbCcgb3JwaGFuIGxvZ2luLiBTaW5jZSB0aGUgZGF0YWJhc2UgaXMgcmVhZCBvbmx5IHdlIGNhbiBub3QgdXNlIHRoZSBzdGFuZGFyZCBmaXggZm9yIG9ycGhhbiB1c2Vycy4NClNvLCByZWFkIFtodHRwczovL3N1cHBvcnQubWljcm9zb2Z0LmNvbS9lbi11cy9rYi85MTg5OTJdKGh0dHBzOi8vc3VwcG9ydC5taWNyb3NvZnQuY29tL2VuLXVzL2tiLzkxODk5Mikgb24gaG93IHRvIGNyZWF0ZSBjbG9uZSBhIFNRTCBTZXJ2ZXIgbG9naW4gZnJvbSBvbmUgaW5zdGFuY2UgdG8gYW5vdGhlciAoYWx0b2dldGhlciB3aXRoIFNJRCBhbmQgcGFzc3dvcmQgaGFzaCku

Wednesday, March 2, 2016

SonarQube for Dummies

This post applies to SonarQube version 5.3 on Windows

Server Install

Download and unzip sonarqube-5.3.zip
Go to bin\windows-x86-64 and run StartSonar.bat (I haven't tried running it as service)
Browse to http://localhost:9000
Go to Administration->System->Update Center->Availabe and install C# (for instance) plug-in
Restart

MSBuild.SonarQube.Runner

Download and unzip MSBuild.SonarQube.Runner-2.0.zip
Add to environment PATH variable MSBuild.SonarQube.Runner-2.0 full path
Install MSBuild 14
Create a batch file to run MSBuild.SonarQube.Runner.exe, something like:

MSBuild.SonarQube.Runner.exe begin /key:fwk /name:Framework /version:1.0
"C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe" /t:Rebuild main\src\Framework\Framework.sln >1.txt
MSBuild.SonarQube.Runner.exe end

Browse to the specified URL (like http://localhost:9000/dashboard/index/fwk)
Happy frustration about all kind of stupid 'major/critical/blocker' issues!

Tuesday, March 1, 2016

Graylog series, part 1

Graylog Primer

This post applies to Graylog version 1.3.3.

Initial Server Install

This is straightforward: download the virtual appliance, import and start. That's it!

Graylog Collector

This post applies to version 0.4.2.

Download graylog-collector-0.4.2.zip archieve and explode it in target directory
Rename collector.conf.example to collector.conf; Modify it according to your needs (there are plenty of samples in Graylog documentation)
Install/start the service running bin\graylog-collector-service.bat install GraylogCollector and bin\graylog-collector-service.bat start GraylogCollector

Now we have a server side problem: by default there are two pre-installed inputs :appliance-gelf-udp and appliance-syslog-udp while the client Collector supports (for GELF at least) only TCP. So we have to create another input according to our configuration (port mainly).

If everything worked according to the plan we should see now in the Search section our events and every second a nginx sourced event (very annoying).

Server insights

Startup scripts

/opt/graylog/service contains symbolic links to all stratup directories. In /etc/init/graylog-runsvdir.conf there is executed exec /opt/graylog/embedded/bin/runsvdir-start which in turn runs exec env - PATH=$PATH runsvdir -P /opt/graylog/service 'log:.

Every directory pointed by the symbolic links mentioned before contains a run script. For instance in /opt/graylog/sv/graylog-server/ we have

#!/bin/sh
exec 2>&1

umask 077

if [ -f "/opt/graylog/embedded/share/graylog/installation-source.sh" ]; then
    . "/opt/graylog/embedded/share/graylog/installation-source.sh"
fi

export JAVA_HOME=/opt/graylog/embedded/jre
export GRAYLOG_SERVER_JAVA_OPTS="-Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"

# check if mongodb is up
timeout 600 bash -c "until curl -s http://127.0.0.1:27017; do sleep 1; done"
exec chpst -P -U graylog -u graylog /opt/graylog/embedded/bin/authbind $JAVA_HOME/bin/java $GRAYLOG_SERVER_JAVA_OPTS -jar -Dlog4j.configuration=file:///opt/graylog/conf/log4j.xml -Djava.library.path=/opt/graylog/server/lib/sigar/ -Dgraylog2.installation_source=${GRAYLOG_INSTALLATION_SOURCE:=unknown} /opt/graylog/server/graylog.jar server -f /opt/graylog/conf/graylog.conf

Logging

Logging is started through log\run. See svloggelfd documentation on github as well as linux svlogd manual. Notice that UDP is used by svloggelfd.

Actual logs are saved in /var/log/graylog/... according to svlogd rules while svloggelfd sends all data to Graylog UDP input.

Sample log\run content:

#!/bin/sh
exec /opt/graylog/embedded/bin/svloggelfd -H 127.0.0.1:12201 -s graylog-server -e | svlogd -tt /var/log/graylog/server

Disabling nginx Loging forward to Graylog

Just cahnge /opt/graylog/sv/nginx/log/run to:

#!/bin/sh
#exec /opt/graylog/embedded/bin/svloggelfd -H 127.0.0.1:12201 -s nginx -e | svlogd -tt /var/log/graylog/nginx
exec svlogd -tt /var/log/graylog/nginx

and restart Graylog (see below)

Controllig Graylog from command line

graylog-ctl status
graylog-ctl stop
graylog-ctl start

Timezone

Use sudo date to change the default settings for date/time (UTC+0).

Changing timezone :

Edit /etc/timezone and change to the desired timezone (like Europe/Bucharest)
Run sudo dpkg-reconfigure --frontend noninteractive tzdata

This is not enough; Graylog web interface will still disply events with time in Etc/UTC timezone.

So:

Check the startup script for graylog-web (semething like exec chpst -P -U graylog -u graylog /opt/graylog/web/bin/graylog-web-interface -Dconfig.file=/opt/graylog/conf/graylog-web-interface.conf -Dhttp.port=9000 -Dhttp.address=0.0.0.0 -Dpidfile.path=/var/opt/graylog/web.pid -Dlogger.file=/opt/graylog/conf/web-logger.xml
Now edit the configuration file (/opt/graylog/conf/graylog-web-interface.conf) and change the timezone property according to your needs
This might not be enough: admin user has a fixed UTC timezone configuration so you will have to create a new user and assign the desired timezone or none (so the default just configured before will be used)

Debugging

This is not about bugs but just inspecting the implementation code to clarify things not captured in official documentation

Server

Change startup script (/opt/graylog/sv/graylog-server/run) to

#!/bin/sh
exec 2>&1

umask 077

if [ -f "/opt/graylog/embedded/share/graylog/installation-source.sh" ]; then
    . "/opt/graylog/embedded/share/graylog/installation-source.sh"
fi

export JAVA_HOME=/opt/graylog/embedded/jre
# export GRAYLOG_SERVER_JAVA_OPTS="-Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"
export GRAYLOG_SERVER_JAVA_OPTS="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=4242 -Xms1g -Xmx1500m -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow"

# check if mongodb is up
timeout 600 bash -c "until curl -s http://127.0.0.1:27017; do sleep 1; done"
exec chpst -P -U graylog -u graylog /opt/graylog/embedded/bin/authbind $JAVA_HOME/bin/java $GRAYLOG_SERVER_JAVA_OPTS -jar -Dlog4j.configuration=file:///opt/graylog/conf/log4j.xml -Djava.library.path=/opt/graylog/server/lib/sigar/ -Dgraylog2.installation_source=${GRAYLOG_INSTALLATION_SOURCE:=unknown} /opt/graylog/server/graylog.jar server -f /opt/graylog/conf/graylog.conf

Download tag 1.3.3 of graylog-server from github
Open pom.xml in your favorite Java IDE
Attach to <graylog-server-fqdn>, port 4242
Happy inspecting!

Web interface

Debugging web interface is a little bit tricky since it is written in Scala-Play without a maven based project, so:

Change the startup script (/opt/graylog/sv/graylog-web/run) to:

#!/bin/sh
exec 2>&1

umask 077
export JAVA_HOME=/opt/graylog/embedded/jre

rm -f /var/opt/graylog/web.pid
exec chpst -P -U graylog -u graylog /opt/graylog/web/bin/graylog-web-interface -jvm-debug 4243 -Dconfig.file=/opt/graylog/conf/graylog-web-interface.conf -Dhttp.port=9000 -Dhttp.address=0.0.0.0 -Dpidfile.path=/var/opt/graylog/web.pid -Dlogger.file=/opt/graylog/conf/web-logger.xml

Notice the -jvm-debug 4243 extra parameter
Configure your favourite Java IDE to use the source path also from graylog2-web-interface-1.3.3\app (download tag 1.3.3 of deprecated web interface sources)
Some functionality refers to classes in graylog2-rest-client module of graylog server project (like org.graylog2.restclient.lib.DateTools containing method getUserTimeZone).
Search is performed in renderSearch method of controllers.SearchController class.
Render template is graylog2-web-interface-1.3.3\app\views\search\index.scala.html
Java script dealing with rendering dates according to the user's timezone is done in app\assets\javascripts\moment-helper.js while gl2UserTimeZone is defined in app\views\partials\navbar.scala.html

IyBHcmF5bG9nIFByaW1lcg0KVGhpcyBwb3N0IGFwcGxpZXMgdG8gR3JheWxvZyB2ZXJzaW9uIDEuMy4zLg0KDQojIyBJbml0aWFsIFNlcnZlciBJbnN0YWxsDQoNClRoaXMgaXMgc3RyYWlnaHRmb3J3YXJkOiBkb3dubG9hZCB0aGUgdmlydHVhbCBhcHBsaWFuY2UsIGltcG9ydCBhbmQgc3RhcnQuIFRoYXQncyBpdCENCg0KIyMgR3JheWxvZyBDb2xsZWN0b3INCg0KVGhpcyBwb3N0IGFwcGxpZXMgdG8gdmVyc2lvbiAwLjQuMi4NCg0KKiBEb3dubG9hZCBncmF5bG9nLWNvbGxlY3Rvci0wLjQuMi56aXAgYXJjaGlldmUgYW5kIGV4cGxvZGUgaXQgaW4gdGFyZ2V0IGRpcmVjdG9yeQ0KKiBSZW5hbWUgYGNvbGxlY3Rvci5jb25mLmV4YW1wbGVgIHRvIGBjb2xsZWN0b3IuY29uZmA7IE1vZGlmeSBpdCBhY2NvcmRpbmcgdG8geW91ciBuZWVkcyAodGhlcmUgYXJlIHBsZW50eSBvZiBzYW1wbGVzIGluIEdyYXlsb2cgZG9jdW1lbnRhdGlvbikNCiogSW5zdGFsbC9zdGFydCB0aGUgc2VydmljZSBydW5uaW5nIGBiaW5cZ3JheWxvZy1jb2xsZWN0b3Itc2VydmljZS5iYXQgaW5zdGFsbCBHcmF5bG9nQ29sbGVjdG9yYCBhbmQgYGJpblxncmF5bG9nLWNvbGxlY3Rvci1zZXJ2aWNlLmJhdCBzdGFydCBHcmF5bG9nQ29sbGVjdG9yYA0KDQpOb3cgd2UgaGF2ZSBhIHNlcnZlciBzaWRlIHByb2JsZW06IGJ5IGRlZmF1bHQgdGhlcmUgYXJlIHR3byBwcmUtaW5zdGFsbGVkIGlucHV0cyA6YGFwcGxpYW5jZS1nZWxmLXVkcGAgYW5kIGBhcHBsaWFuY2Utc3lzbG9nLXVkcGAgd2hpbGUgdGhlIGNsaWVudCBDb2xsZWN0b3Igc3VwcG9ydHMgKGZvciBHRUxGIGF0IGxlYXN0KSBvbmx5IFRDUC4gU28gd2UgaGF2ZSB0byBjcmVhdGUgYW5vdGhlciBpbnB1dCBhY2NvcmRpbmcgdG8gb3VyIGNvbmZpZ3VyYXRpb24gKGBwb3J0YCBtYWlubHkpLg0KDQpJZiBldmVyeXRoaW5nIHdvcmtlZCBhY2NvcmRpbmcgdG8gdGhlIHBsYW4gd2Ugc2hvdWxkIHNlZSBub3cgaW4gdGhlIFNlYXJjaCBzZWN0aW9uIG91ciBldmVudHMgYW5kIGV2ZXJ5IHNlY29uZCBhIGBuZ2lueGAgc291cmNlZCBldmVudCAodmVyeSBhbm5veWluZykuDQoNCiMjIFNlcnZlciBpbnNpZ2h0cw0KDQojIyMgU3RhcnR1cCBzY3JpcHRzDQoNCmAvb3B0L2dyYXlsb2cvc2VydmljZWAgY29udGFpbnMgc3ltYm9saWMgbGlua3MgdG8gYWxsIHN0cmF0dXAgZGlyZWN0b3JpZXMuIEluIGAvZXRjL2luaXQvZ3JheWxvZy1ydW5zdmRpci5jb25mYCB0aGVyZSBpcyBleGVjdXRlZCBgZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3J1bnN2ZGlyLXN0YXJ0YCB3aGljaCBpbiB0dXJuIHJ1bnMgYGV4ZWMgZW52IC0gUEFUSD0kUEFUSCBydW5zdmRpciAtUCAvb3B0L2dyYXlsb2cvc2VydmljZSAnbG9nOmAuDQoNCkV2ZXJ5IGRpcmVjdG9yeSBwb2ludGVkIGJ5IHRoZSBzeW1ib2xpYyBsaW5rcyBtZW50aW9uZWQgYmVmb3JlIGNvbnRhaW5zIGEgYHJ1bmAgc2NyaXB0LiBGb3IgaW5zdGFuY2UgaW4gYC9vcHQvZ3JheWxvZy9zdi9ncmF5bG9nLXNlcnZlci9gIHdlIGhhdmUNCg0KYGBgYmFzaA0KIyEvYmluL3NoDQpleGVjIDI+JjENCg0KdW1hc2sgMDc3DQoNCmlmIFsgLWYgIi9vcHQvZ3JheWxvZy9lbWJlZGRlZC9zaGFyZS9ncmF5bG9nL2luc3RhbGxhdGlvbi1zb3VyY2Uuc2giIF07IHRoZW4NCiAgICAuICIvb3B0L2dyYXlsb2cvZW1iZWRkZWQvc2hhcmUvZ3JheWxvZy9pbnN0YWxsYXRpb24tc291cmNlLnNoIg0KZmkNCg0KZXhwb3J0IEpBVkFfSE9NRT0vb3B0L2dyYXlsb2cvZW1iZWRkZWQvanJlDQpleHBvcnQgR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTPSItWG1zMWcgLVhteDE1MDBtIC1YWDpOZXdSYXRpbz0xIC1YWDpQZXJtU2l6ZT0xMjhtIC1YWDpNYXhQZXJtU2l6ZT0yNTZtIC1zZXJ2ZXIgLVhYOitSZXNpemVUTEFCIC1YWDorVXNlQ29uY01hcmtTd2VlcEdDIC1YWDorQ01TQ29uY3VycmVudE1URW5hYmxlZCAtWFg6K0NNU0NsYXNzVW5sb2FkaW5nRW5hYmxlZCAtWFg6K1VzZVBhck5ld0dDIC1YWDotT21pdFN0YWNrVHJhY2VJbkZhc3RUaHJvdyINCg0KIyBjaGVjayBpZiBtb25nb2RiIGlzIHVwDQp0aW1lb3V0IDYwMCBiYXNoIC1jICJ1bnRpbCBjdXJsIC1zIGh0dHA6Ly8xMjcuMC4wLjE6MjcwMTc7IGRvIHNsZWVwIDE7IGRvbmUiDQpleGVjIGNocHN0IC1QIC1VIGdyYXlsb2cgLXUgZ3JheWxvZyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL2F1dGhiaW5kICRKQVZBX0hPTUUvYmluL2phdmEgJEdSQVlMT0dfU0VSVkVSX0pBVkFfT1BUUyAtamFyIC1EbG9nNGouY29uZmlndXJhdGlvbj1maWxlOi8vL29wdC9ncmF5bG9nL2NvbmYvbG9nNGoueG1sIC1EamF2YS5saWJyYXJ5LnBhdGg9L29wdC9ncmF5bG9nL3NlcnZlci9saWIvc2lnYXIvIC1EZ3JheWxvZzIuaW5zdGFsbGF0aW9uX3NvdXJjZT0ke0dSQVlMT0dfSU5TVEFMTEFUSU9OX1NPVVJDRTo9dW5rbm93bn0gL29wdC9ncmF5bG9nL3NlcnZlci9ncmF5bG9nLmphciBzZXJ2ZXIgLWYgL29wdC9ncmF5bG9nL2NvbmYvZ3JheWxvZy5jb25mDQpgYGANCg0KIyMjIExvZ2dpbmcNCg0KTG9nZ2luZyBpcyBzdGFydGVkIHRocm91Z2ggYGxvZ1xydW5gLiBTZWUgYHN2bG9nZ2VsZmRgIGRvY3VtZW50YXRpb24gb24gZ2l0aHViIGFzIHdlbGwgYXMgbGludXggYHN2bG9nZGAgbWFudWFsLiBOb3RpY2UgdGhhdCBVRFAgaXMgdXNlZCBieSBgc3Zsb2dnZWxmZGAuDQoNCkFjdHVhbCBsb2dzIGFyZSBzYXZlZCBpbiBgL3Zhci9sb2cvZ3JheWxvZy8uLi5gIGFjY29yZGluZyB0byBgc3Zsb2dkYCBydWxlcyB3aGlsZSBgc3Zsb2dnZWxmZGAgc2VuZHMgYWxsIGRhdGEgdG8gR3JheWxvZyBVRFAgaW5wdXQuDQoNClNhbXBsZSBgbG9nXHJ1bmAgY29udGVudDoNCmBgYGJhc2gNCiMhL2Jpbi9zaA0KZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3N2bG9nZ2VsZmQgLUggMTI3LjAuMC4xOjEyMjAxIC1zIGdyYXlsb2ctc2VydmVyIC1lIHwgc3Zsb2dkIC10dCAvdmFyL2xvZy9ncmF5bG9nL3NlcnZlcg0KYGBgDQoNCiMjIyBEaXNhYmxpbmcgbmdpbnggTG9naW5nIGZvcndhcmQgdG8gR3JheWxvZw0KDQpKdXN0IGNhaG5nZSBgL29wdC9ncmF5bG9nL3N2L25naW54L2xvZy9ydW5gIHRvOg0KYGBgYmFzaA0KIyEvYmluL3NoDQojZXhlYyAvb3B0L2dyYXlsb2cvZW1iZWRkZWQvYmluL3N2bG9nZ2VsZmQgLUggMTI3LjAuMC4xOjEyMjAxIC1zIG5naW54IC1lIHwgc3Zsb2dkIC10dCAvdmFyL2xvZy9ncmF5bG9nL25naW54DQpleGVjIHN2bG9nZCAtdHQgL3Zhci9sb2cvZ3JheWxvZy9uZ2lueA0KYGBgDQphbmQgcmVzdGFydCBHcmF5bG9nIChzZWUgYmVsb3cpDQoNCiMjIyBDb250cm9sbGlnIEdyYXlsb2cgZnJvbSBjb21tYW5kIGxpbmUNCg0KYGBgYmFzaA0KZ3JheWxvZy1jdGwgc3RhdHVzDQpncmF5bG9nLWN0bCBzdG9wDQpncmF5bG9nLWN0bCBzdGFydA0KYGBgDQoNCiMjIyBUaW1lem9uZQ0KDQpVc2UgYHN1ZG8gZGF0ZWAgdG8gY2hhbmdlIHRoZSBkZWZhdWx0IHNldHRpbmdzIGZvciBkYXRlL3RpbWUgKFVUQyswKS4NCg0KQ2hhbmdpbmcgdGltZXpvbmUgOg0KKiBFZGl0IC9ldGMvdGltZXpvbmUgYW5kIGNoYW5nZSB0byB0aGUgZGVzaXJlZCB0aW1lem9uZSAobGlrZSBFdXJvcGUvQnVjaGFyZXN0KQ0KKiBSdW4gc3VkbyBkcGtnLXJlY29uZmlndXJlIC0tZnJvbnRlbmQgbm9uaW50ZXJhY3RpdmUgdHpkYXRhDQoNClRoaXMgaXMgbm90IGVub3VnaDsgR3JheWxvZyB3ZWIgaW50ZXJmYWNlIHdpbGwgc3RpbGwgZGlzcGx5IGV2ZW50cyB3aXRoIHRpbWUgaW4gRXRjL1VUQyB0aW1lem9uZS4NCg0KU286DQoqIENoZWNrIHRoZSBzdGFydHVwIHNjcmlwdCBmb3IgZ3JheWxvZy13ZWIgKHNlbWV0aGluZyBsaWtlIGBleGVjIGNocHN0IC1QIC1VIGdyYXlsb2cgLXUgZ3JheWxvZyAvb3B0L2dyYXlsb2cvd2ViL2Jpbi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UgLURjb25maWcuZmlsZT0vb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UuY29uZiAtRGh0dHAucG9ydD05MDAwIC1EaHR0cC5hZGRyZXNzPTAuMC4wLjAgLURwaWRmaWxlLnBhdGg9L3Zhci9vcHQvZ3JheWxvZy93ZWIucGlkIC1EbG9nZ2VyLmZpbGU9L29wdC9ncmF5bG9nL2NvbmYvd2ViLWxvZ2dlci54bWxgDQoqIE5vdyBlZGl0IHRoZSBjb25maWd1cmF0aW9uIGZpbGUgKGAvb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLXdlYi1pbnRlcmZhY2UuY29uZmApIGFuZCBjaGFuZ2UgdGhlIHRpbWV6b25lIHByb3BlcnR5IGFjY29yZGluZyB0byB5b3VyIG5lZWRzDQoqIFRoaXMgbWlnaHQgbm90IGJlIGVub3VnaDogYWRtaW4gdXNlciBoYXMgYSBmaXhlZCBVVEMgdGltZXpvbmUgY29uZmlndXJhdGlvbiBzbyB5b3Ugd2lsbCBoYXZlIHRvIGNyZWF0ZSBhIG5ldyB1c2VyIGFuZCBhc3NpZ24gdGhlIGRlc2lyZWQgdGltZXpvbmUgb3Igbm9uZSAoc28gdGhlIGRlZmF1bHQganVzdCBjb25maWd1cmVkIGJlZm9yZSB3aWxsIGJlIHVzZWQpDQoNCiMjIyBEZWJ1Z2dpbmcNCg0KVGhpcyBpcyBub3QgYWJvdXQgYnVncyBidXQganVzdCBpbnNwZWN0aW5nIHRoZSBpbXBsZW1lbnRhdGlvbiBjb2RlIHRvIGNsYXJpZnkgdGhpbmdzIG5vdCBjYXB0dXJlZCBpbiBvZmZpY2lhbCBkb2N1bWVudGF0aW9uDQoNCiMjIyMgU2VydmVyDQoNCiogQ2hhbmdlIHN0YXJ0dXAgc2NyaXB0IChgL29wdC9ncmF5bG9nL3N2L2dyYXlsb2ctc2VydmVyL3J1bmApIHRvDQpgYGBiYXNoDQojIS9iaW4vc2gNCmV4ZWMgMj4mMQ0KDQp1bWFzayAwNzcNCg0KaWYgWyAtZiAiL29wdC9ncmF5bG9nL2VtYmVkZGVkL3NoYXJlL2dyYXlsb2cvaW5zdGFsbGF0aW9uLXNvdXJjZS5zaCIgXTsgdGhlbg0KICAgIC4gIi9vcHQvZ3JheWxvZy9lbWJlZGRlZC9zaGFyZS9ncmF5bG9nL2luc3RhbGxhdGlvbi1zb3VyY2Uuc2giDQpmaQ0KDQpleHBvcnQgSkFWQV9IT01FPS9vcHQvZ3JheWxvZy9lbWJlZGRlZC9qcmUNCiMgZXhwb3J0IEdSQVlMT0dfU0VSVkVSX0pBVkFfT1BUUz0iLVhtczFnIC1YbXgxNTAwbSAtWFg6TmV3UmF0aW89MSAtWFg6UGVybVNpemU9MTI4bSAtWFg6TWF4UGVybVNpemU9MjU2bSAtc2VydmVyIC1YWDorUmVzaXplVExBQiAtWFg6K1VzZUNvbmNNYXJrU3dlZXBHQyAtWFg6K0NNU0NvbmN1cnJlbnRNVEVuYWJsZWQgLVhYOitDTVNDbGFzc1VubG9hZGluZ0VuYWJsZWQgLVhYOitVc2VQYXJOZXdHQyAtWFg6LU9taXRTdGFja1RyYWNlSW5GYXN0VGhyb3ciDQpleHBvcnQgR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTPSItWGRlYnVnIC1Ybm9hZ2VudCAtWHJ1bmpkd3A6dHJhbnNwb3J0PWR0X3NvY2tldCxzZXJ2ZXI9eSxzdXNwZW5kPW4sYWRkcmVzcz00MjQyIC1YbXMxZyAtWG14MTUwMG0gLVhYOk5ld1JhdGlvPTEgLVhYOlBlcm1TaXplPTEyOG0gLVhYOk1heFBlcm1TaXplPTI1Nm0gLXNlcnZlciAtWFg6K1Jlc2l6ZVRMQUIgLVhYOitVc2VDb25jTWFya1N3ZWVwR0MgLVhYOitDTVNDb25jdXJyZW50TVRFbmFibGVkIC1YWDorQ01TQ2xhc3NVbmxvYWRpbmdFbmFibGVkIC1YWDorVXNlUGFyTmV3R0MgLVhYOi1PbWl0U3RhY2tUcmFjZUluRmFzdFRocm93Ig0KDQojIGNoZWNrIGlmIG1vbmdvZGIgaXMgdXANCnRpbWVvdXQgNjAwIGJhc2ggLWMgInVudGlsIGN1cmwgLXMgaHR0cDovLzEyNy4wLjAuMToyNzAxNzsgZG8gc2xlZXAgMTsgZG9uZSINCmV4ZWMgY2hwc3QgLVAgLVUgZ3JheWxvZyAtdSBncmF5bG9nIC9vcHQvZ3JheWxvZy9lbWJlZGRlZC9iaW4vYXV0aGJpbmQgJEpBVkFfSE9NRS9iaW4vamF2YSAkR1JBWUxPR19TRVJWRVJfSkFWQV9PUFRTIC1qYXIgLURsb2c0ai5jb25maWd1cmF0aW9uPWZpbGU6Ly8vb3B0L2dyYXlsb2cvY29uZi9sb2c0ai54bWwgLURqYXZhLmxpYnJhcnkucGF0aD0vb3B0L2dyYXlsb2cvc2VydmVyL2xpYi9zaWdhci8gLURncmF5bG9nMi5pbnN0YWxsYXRpb25fc291cmNlPSR7R1JBWUxPR19JTlNUQUxMQVRJT05fU09VUkNFOj11bmtub3dufSAvb3B0L2dyYXlsb2cvc2VydmVyL2dyYXlsb2cuamFyIHNlcnZlciAtZiAvb3B0L2dyYXlsb2cvY29uZi9ncmF5bG9nLmNvbmYNCmBgYA0KKiBEb3dubG9hZCB0YWcgMS4zLjMgb2YgZ3JheWxvZy1zZXJ2ZXIgZnJvbSBnaXRodWINCiogT3BlbiBwb20ueG1sIGluIHlvdXIgZmF2b3JpdGUgSmF2YSBJREUNCiogQXR0YWNoIHRvIDxncmF5bG9nLXNlcnZlci1mcWRuPiwgcG9ydCA0MjQyDQoqIEhhcHB5IGluc3BlY3RpbmchDQoNCiMjIyMgV2ViIGludGVyZmFjZQ0KDQpEZWJ1Z2dpbmcgd2ViIGludGVyZmFjZSBpcyBhIGxpdHRsZSBiaXQgdHJpY2t5IHNpbmNlIGl0IGlzIHdyaXR0ZW4gaW4gU2NhbGEtUGxheSB3aXRob3V0IGEgbWF2ZW4gYmFzZWQgcHJvamVjdCwgc286DQoNCiogQ2hhbmdlIHRoZSBzdGFydHVwIHNjcmlwdCAoYC9vcHQvZ3JheWxvZy9zdi9ncmF5bG9nLXdlYi9ydW5gKSB0bzoNCmBgYGJhc2gNCiMhL2Jpbi9zaA0KZXhlYyAyPiYxDQoNCnVtYXNrIDA3Nw0KZXhwb3J0IEpBVkFfSE9NRT0vb3B0L2dyYXlsb2cvZW1iZWRkZWQvanJlDQoNCnJtIC1mIC92YXIvb3B0L2dyYXlsb2cvd2ViLnBpZA0KZXhlYyBjaHBzdCAtUCAtVSBncmF5bG9nIC11IGdyYXlsb2cgL29wdC9ncmF5bG9nL3dlYi9iaW4vZ3JheWxvZy13ZWItaW50ZXJmYWNlIC1qdm0tZGVidWcgNDI0MyAtRGNvbmZpZy5maWxlPS9vcHQvZ3JheWxvZy9jb25mL2dyYXlsb2ctd2ViLWludGVyZmFjZS5jb25mIC1EaHR0cC5wb3J0PTkwMDAgLURodHRwLmFkZHJlc3M9MC4wLjAuMCAtRHBpZGZpbGUucGF0aD0vdmFyL29wdC9ncmF5bG9nL3dlYi5waWQgLURsb2dnZXIuZmlsZT0vb3B0L2dyYXlsb2cvY29uZi93ZWItbG9nZ2VyLnhtbA0KYGBgDQoqIE5vdGljZSB0aGUgYC1qdm0tZGVidWcgNDI0M2AgZXh0cmEgcGFyYW1ldGVyDQoqIENvbmZpZ3VyZSB5b3VyIGZhdm91cml0ZSBKYXZhIElERSB0byB1c2UgdGhlIHNvdXJjZSBwYXRoIGFsc28gZnJvbSBgZ3JheWxvZzItd2ViLWludGVyZmFjZS0xLjMuM1xhcHBgIChkb3dubG9hZCB0YWcgMS4zLjMgb2YgZGVwcmVjYXRlZCB3ZWIgaW50ZXJmYWNlIHNvdXJjZXMpDQoqIFNvbWUgZnVuY3Rpb25hbGl0eSByZWZlcnMgdG8gY2xhc3NlcyBpbiBgZ3JheWxvZzItcmVzdC1jbGllbnRgIG1vZHVsZSBvZiBncmF5bG9nIHNlcnZlciBwcm9qZWN0ICAobGlrZSBgb3JnLmdyYXlsb2cyLnJlc3RjbGllbnQubGliLkRhdGVUb29sc2AgY29udGFpbmluZyBtZXRob2QgYGdldFVzZXJUaW1lWm9uZWApLg0KKiBTZWFyY2ggaXMgcGVyZm9ybWVkIGluIGByZW5kZXJTZWFyY2hgIG1ldGhvZCBvZiBgY29udHJvbGxlcnMuU2VhcmNoQ29udHJvbGxlcmAgY2xhc3MuDQoqIFJlbmRlciB0ZW1wbGF0ZSBpcyBgZ3JheWxvZzItd2ViLWludGVyZmFjZS0xLjMuM1xhcHBcdmlld3Ncc2VhcmNoXGluZGV4LnNjYWxhLmh0bWxgDQoqIEphdmEgc2NyaXB0IGRlYWxpbmcgd2l0aCByZW5kZXJpbmcgZGF0ZXMgYWNjb3JkaW5nIHRvIHRoZSB1c2VyJ3MgdGltZXpvbmUgaXMgZG9uZSBpbiBgYXBwXGFzc2V0c1xqYXZhc2NyaXB0c1xtb21lbnQtaGVscGVyLmpzYCB3aGlsZSBgZ2wyVXNlclRpbWVab25lYCBpcyBkZWZpbmVkIGluIGBhcHBcdmlld3NccGFydGlhbHNcbmF2YmFyLnNjYWxhLmh0bWxg

Subscribe to: Posts ( Atom )