This post applies to:
- Jetty version 9.4.6.v20170531
- Spring Boot 1.5.6.RELEASE
- Ubuntu 16.04.3 LTS
The problem
Running Spring Boot applications in Jetty environment as WARs is not as simple as the documentation states due to:
The main requirement was to run some simple Spring Boot application that performs a scheduled task.
The main problem was that after installing Jetty and deploying the WAR nothing happened…Ups!
The Resolution
Step 1
The Spring Boot application was using the following entry point:
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.support.SpringBootServletInitializer;
import org.springframework.context.annotation.Bean;
import org.springframework.scheduling.TaskScheduler;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.concurrent.ConcurrentTaskScheduler;
@SpringBootApplication
@EnableScheduling
public class SalusptApplication extends SpringBootServletInitializer {
@Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(SalusptApplication.class);
}
public static void main(String[] args) {
SpringApplication.run(SalusptApplication.class, args);
}
@Bean
public TaskScheduler taskScheduler() {
return new ConcurrentTaskScheduler();
}
}
At this point application started to run properly from IDE (STS) so it was the time to deploy it on Jetty: run mvn package
and you obtain the WAR under two flavours:
- target<application>.WAR
- target<application>-original.war
The first one contains also the jars for running with embeded application server but you can safely deploy it.
Remark: Jetty was running as root
since no JETTY_USER
environment variable was defined yet.
Deployment looked successfull (no errors in logs) but nothing happened.
After some trial/error attempts it was quite obvious that configure
method was not called so no chance for Spring Boot initialization process to be triggered. Nothing in official Spring Boot reference was mentioning about enabling anotations
module on Jetty, of course we have had to be more attentive in our kindergarten Java courses…(I am a newbie in Spring/Jetty world, why not to find out things the hard way!)
Step 2
After enabling the annotation
module another surprise…application still not working: I’ve got (by analysing logs) an error similar to the one described here: https://stackoverflow.com/questions/41995029/jersey-spring3–2–25–1-produces-failed-startup-of-context-error-in-jetty–9–3. Removing Jersey dependency in Maven pom.xml solved the problem (somehow, at some point in future I plan to have some REST API exposed to the world, fingers cross!)
Step 3
Without really understanding what I was doing at the phase of generatig the project scheleton through https://start.spring.io/ I added devtools
to the project. Well, trying to deploy to Jetty the WAR I’ve got another error, something with not finding an entry point for restarting the application. Till restarting I neede a first proper start so…good by devtools
.
Step 4
Now added JETTY_USER=jetty
to /env/defaults/jetty
Another surprise…Jetty would not start. After another late night hours of trial and error attempts I noticed that if I change ownership of \run\jetty to user jetty the server starts and I can deploy the WAR. The problem was that after restrating the server the permissions on /run directory were lost.
Ok, time to analyze /etc/init.d/jetty
. The short story (also described here: https://stackoverflow.com/questions/17999729/jetty-bash-script-works-only-with-root-user):
The evil hides behind the following snippet:
if [ -z "$JETTY_RUN" ]
then
JETTY_RUN=$(findDirectory -w /var/run /usr/var/run $JETTY_BASE /tmp)/jetty
[ -d "$JETTY_RUN" ] || mkdir $JETTY_RUN
fi
If no JETTY_RUN
environment variable is defined the script attempts to build a base one and appends jetty
to it; so the directory is created (surprise) under root
account and later attempt to start jetty (start-stop-daemon -S -p"$JETTY_PID" $CH_USER -d"$JETTY_BASE" -b -m -a "$JAVA" -- "${RUN_ARGS[@]}" start-log-file="$JETTY_START_LOG"
) with JETTY_START_LOG="$JETTY_RUN/$NAME-start.log"
will fail since jetty user will not be able to create the log file.
The solution was to pre-define the JETTY_RUN
environment variable pointing to a directory where jetty is owner.
Appendix
Full Jetty installation steps
Java 8
sudo -i
Run one by one the following commands;watch out for capitalized Y
on answers.
apt install software-properties-common
add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer
#apt-get install oracle-java8-set-default
Download and install Jetty
sudo -i
mkdir -p /opt/jetty
mkdir -p /opt/web/jettybase
mkdir /opt/web/jettybase/run
mkdir -p /opt/jetty/temp
useradd --user-group --shell /bin/false --home-dir /opt/jetty/temp jetty
chown -R jetty:jetty /opt/jetty
chown -R jetty:jetty /opt/web/jettybase
chown -R jetty:jetty /opt/jetty/temp
cd /tmp
wget http://central.maven.org/maven2/org/eclipse/jetty/jetty-distribution/9.4.6.v20170531/jetty-distribution-9.4.6.v20170531.tar.gz
cd /opt/jetty
tar -zxf /tmp/jetty-distribution-9.4.6.v20170531.tar.gz
cd /opt/jetty/jetty-distribution-9.4.6.v20170531
cp bin/jetty.sh /etc/init.d/jetty
cd /opt/web/jettybase
java -jar /opt/jetty/jetty-distribution-9.4.6.v20170531/start.jar --add-to-start=deploy,http,console-capture,annotations
#configures Java for debug, must be removed from production environmemnts
echo "JETTY_HOME=/opt/jetty/jetty-distribution-9.4.6.v20170531" > /etc/default/jetty
echo "JETTY_BASE=/opt/web/jettybase" >> /etc/default/jetty
echo "JETTY_RUN=/opt/web/jettybase/run" >> /etc/default/jetty
echo "TMPDIR=/opt/jetty/temp" >> /etc/default/jetty
echo "JAVA_OPTIONS=\"-Xdebug -agentlib:jdwp=transport=dt_socket,address=9999,server=y,suspend=n\"" >> /etc/default/jetty
echo "JETTY_USER=jetty" >> /etc/default/jetty
service jetty check
update-rc.d jetty defaults
service jetty start
service jetty status
# if service not starting check all involved directories that jetty user is owner
# just for development, in order to be able to copy files through WinSCP:
chmod o+w /opt/web/jettybase/webapps
How Spring Boot Bootsrap Should Work
See also http://piotrnowicki.com/2011/03/using-servlets–3–0-servletcontainerinitializer/
The >=3.0 Servlet Container (Jetty in our case) will try (if annotations module is activated!) to find in class path all implementations of javax.servlet.ServletContainerInitializer
and will call its onStartup
method with a Set
of all implementations of the @HandlesTypes
mentioned class:
package org.springframework.web;
...
@HandlesTypes(WebApplicationInitializer.class)
public class SpringServletContainerInitializer implements ServletContainerInitializer {
/**
* Delegate the {@code ServletContext} to any {@link WebApplicationInitializer}
* implementations present on the application classpath.
* <p>Because this class declares @{@code HandlesTypes(WebApplicationInitializer.class)},
* Servlet 3.0+ containers will automatically scan the classpath for implementations
* of Spring's {@code WebApplicationInitializer} interface and provide the set of all
* such types to the {@code webAppInitializerClasses} parameter of this method.
* <p>If no {@code WebApplicationInitializer} implementations are found on the classpath,
* this method is effectively a no-op. An INFO-level log message will be issued notifying
* the user that the {@code ServletContainerInitializer} has indeed been invoked but that
* no {@code WebApplicationInitializer} implementations were found.
* <p>Assuming that one or more {@code WebApplicationInitializer} types are detected,
* they will be instantiated (and <em>sorted</em> if the @{@link
* org.springframework.core.annotation.Order @Order} annotation is present or
* the {@link org.springframework.core.Ordered Ordered} interface has been
* implemented). Then the {@link WebApplicationInitializer#onStartup(ServletContext)}
* method will be invoked on each instance, delegating the {@code ServletContext} such
* that each instance may register and configure servlets such as Spring's
* {@code DispatcherServlet}, listeners such as Spring's {@code ContextLoaderListener},
* or any other Servlet API componentry such as filters.
* @param webAppInitializerClasses all implementations of
* {@link WebApplicationInitializer} found on the application classpath
* @param servletContext the servlet context to be initialized
* @see WebApplicationInitializer#onStartup(ServletContext)
* @see AnnotationAwareOrderComparator
*/
@Override
public void onStartup(Set<Class<?>> webAppInitializerClasses, ServletContext servletContext)
throws ServletException {
List<WebApplicationInitializer> initializers = new LinkedList<WebApplicationInitializer>();
if (webAppInitializerClasses != null) {
for (Class<?> waiClass : webAppInitializerClasses) {
// Be defensive: Some servlet containers provide us with invalid classes,
// no matter what @HandlesTypes says...
...
PHA+VGhpcyBwb3N0IGFwcGxpZXMgdG86PC9wPg0KDQo8dWw+DQo8bGk+SmV0dHkgdmVyc2lv
biA5LjQuNi52MjAxNzA1MzE8L2xpPg0KPGxpPlNwcmluZyBCb290IDEuNS42LlJFTEVBU0U8
L2xpPg0KPGxpPlVidW50dSAxNi4wNC4zIExUUzwvbGk+DQo8L3VsPg0KDQo8aDEgaWQ9InRo
ZXByb2JsZW0iPlRoZSBwcm9ibGVtPC9oMT4NCg0KPHA+UnVubmluZyBTcHJpbmcgQm9vdCBh
cHBsaWNhdGlvbnMgaW4gSmV0dHkgZW52aXJvbm1lbnQgYXMgV0FScyBpcyBub3QgYXMgc2lt
cGxlIGFzIHRoZSBkb2N1bWVudGF0aW9uIHN0YXRlcyBkdWUgdG86PC9wPg0KDQo8dWw+DQo8
bGk+TGFjayBvZiBkb2N1bWVudGF0aW9uIGFib3V0IGluc3RhbGxpbmcgSmV0dHkgKHNwZWNp
ZmllZCB2ZXJzaW9uKSBvbiBVYnVudHUgKHNwZWNpZmllZCB2ZXJzaW9uKSAoc29tZSBzYW1w
bGVzIEkgdXNlZCk6DQoNCjx1bD4NCjxsaT48YSBocmVmPSJodHRwOi8vd3d3LnVidW50dWdl
ZWsuY29tL2luc3RhbGwtamV0dHktOS1qYXZhLXNlcnZsZXQtZW5naW5lLWFuZC13ZWJzZXJ2
ZXItb24tdWJ1bnR1LTE1LTA0LXNlcnZlci5odG1sIj5odHRwOi8vd3d3LnVidW50dWdlZWsu
Y29tL2luc3RhbGwtamV0dHkmIzgyMTE7OS1qYXZhLXNlcnZsZXQtZW5naW5lLWFuZC13ZWJz
ZXJ2ZXItb24tdWJ1bnR1JiM4MjExOzE1JiM4MjExOzA0LXNlcnZlci5odG1sPC9hPjwvbGk+
DQo8bGk+PGEgaHJlZj0iaHR0cHM6Ly9ob3N0cHJlc3RvLmNvbS9jb21tdW5pdHkvdHV0b3Jp
YWxzL2hvdy10by1pbnN0YWxsLWpldHR5LTktb24tdWJ1bnR1LTE0LTA0LyI+aHR0cHM6Ly9o
b3N0cHJlc3RvLmNvbS9jb21tdW5pdHkvdHV0b3JpYWxzL2hvdy10by1pbnN0YWxsLWpldHR5
JiM4MjExOzktb24tdWJ1bnR1JiM4MjExOzE0JiM4MjExOzA0LzwvYT48L2xpPg0KPC91bD48
L2xpPg0KPGxpPkNvbXBsZXRlbHkgb3V0ZGF0ZWQvd3JvbmcgSmV0dHkgaW5zdGFsbGF0aW9u
IGluc3RydWN0aW9ucyAoPGEgaHJlZj0iaHR0cHM6Ly93d3cuZWNsaXBzZS5vcmcvamV0dHkv
ZG9jdW1lbnRhdGlvbi85LjQueC9zdGFydHVwLXVuaXgtc2VydmljZS5odG1sIj5odHRwczov
L3d3dy5lY2xpcHNlLm9yZy9qZXR0eS9kb2N1bWVudGF0aW9uLzkuNC54L3N0YXJ0dXAtdW5p
eC1zZXJ2aWNlLmh0bWw8L2E+KTwvbGk+DQo8L3VsPg0KDQo8cD5UaGUgbWFpbiByZXF1aXJl
bWVudCB3YXMgdG8gcnVuIHNvbWUgc2ltcGxlIFNwcmluZyBCb290IGFwcGxpY2F0aW9uIHRo
YXQgcGVyZm9ybXMgYSBzY2hlZHVsZWQgdGFzay4NClRoZSBtYWluIHByb2JsZW0gd2FzIHRo
YXQgYWZ0ZXIgaW5zdGFsbGluZyBKZXR0eSBhbmQgZGVwbG95aW5nIHRoZSBXQVIgbm90aGlu
ZyBoYXBwZW5lZCYjODIzMDtVcHMhPC9wPg0KDQo8aDEgaWQ9InRoZXJlc29sdXRpb24iPlRo
ZSBSZXNvbHV0aW9uPC9oMT4NCg0KPGgyIGlkPSJzdGVwMSI+U3RlcCAxPC9oMj4NCg0KPHA+
VGhlIFNwcmluZyBCb290IGFwcGxpY2F0aW9uIHdhcyB1c2luZyB0aGUgZm9sbG93aW5nIGVu
dHJ5IHBvaW50OjwvcD4NCg0KPHByZT48Y29kZSBjbGFzcz0iamF2YSI+aW1wb3J0IG9yZy5z
cHJpbmdmcmFtZXdvcmsuYm9vdC5TcHJpbmdBcHBsaWNhdGlvbjsNCmltcG9ydCBvcmcuc3By
aW5nZnJhbWV3b3JrLmJvb3QuYXV0b2NvbmZpZ3VyZS5TcHJpbmdCb290QXBwbGljYXRpb247
DQppbXBvcnQgb3JnLnNwcmluZ2ZyYW1ld29yay5ib290LmJ1aWxkZXIuU3ByaW5nQXBwbGlj
YXRpb25CdWlsZGVyOw0KaW1wb3J0IG9yZy5zcHJpbmdmcmFtZXdvcmsuYm9vdC53ZWIuc3Vw
cG9ydC5TcHJpbmdCb290U2VydmxldEluaXRpYWxpemVyOw0KaW1wb3J0IG9yZy5zcHJpbmdm
cmFtZXdvcmsuY29udGV4dC5hbm5vdGF0aW9uLkJlYW47DQppbXBvcnQgb3JnLnNwcmluZ2Zy
YW1ld29yay5zY2hlZHVsaW5nLlRhc2tTY2hlZHVsZXI7DQppbXBvcnQgb3JnLnNwcmluZ2Zy
YW1ld29yay5zY2hlZHVsaW5nLmFubm90YXRpb24uRW5hYmxlU2NoZWR1bGluZzsNCmltcG9y
dCBvcmcuc3ByaW5nZnJhbWV3b3JrLnNjaGVkdWxpbmcuY29uY3VycmVudC5Db25jdXJyZW50
VGFza1NjaGVkdWxlcjsNCg0KQFNwcmluZ0Jvb3RBcHBsaWNhdGlvbg0KQEVuYWJsZVNjaGVk
dWxpbmcNCnB1YmxpYyBjbGFzcyBTYWx1c3B0QXBwbGljYXRpb24gZXh0ZW5kcyBTcHJpbmdC
b290U2VydmxldEluaXRpYWxpemVyIHsNCg0KICAgIEBPdmVycmlkZQ0KICAgIHByb3RlY3Rl
ZCBTcHJpbmdBcHBsaWNhdGlvbkJ1aWxkZXIgY29uZmlndXJlKFNwcmluZ0FwcGxpY2F0aW9u
QnVpbGRlciBhcHBsaWNhdGlvbikgew0KICAgICAgICByZXR1cm4gYXBwbGljYXRpb24uc291
cmNlcyhTYWx1c3B0QXBwbGljYXRpb24uY2xhc3MpOw0KICAgIH0NCg0KICAgIHB1YmxpYyBz
dGF0aWMgdm9pZCBtYWluKFN0cmluZ1tdIGFyZ3MpIHsNCiAgICAgICAgU3ByaW5nQXBwbGlj
YXRpb24ucnVuKFNhbHVzcHRBcHBsaWNhdGlvbi5jbGFzcywgYXJncyk7DQogICAgfQ0KICAg
IA0KICAgIEBCZWFuDQogICAgcHVibGljIFRhc2tTY2hlZHVsZXIgdGFza1NjaGVkdWxlcigp
IHsNCiAgICAgICAgcmV0dXJuIG5ldyBDb25jdXJyZW50VGFza1NjaGVkdWxlcigpOw0KICAg
IH0NCiAgICANCn0NCjwvY29kZT48L3ByZT4NCg0KPHA+QXQgdGhpcyBwb2ludCBhcHBsaWNh
dGlvbiBzdGFydGVkIHRvIHJ1biBwcm9wZXJseSBmcm9tIElERSAoU1RTKSBzbyBpdCB3YXMg
dGhlIHRpbWUgdG8gZGVwbG95IGl0IG9uIEpldHR5OiBydW4gPGNvZGU+bXZuIHBhY2thZ2U8
L2NvZGU+IGFuZCB5b3Ugb2J0YWluIHRoZSBXQVIgdW5kZXIgdHdvIGZsYXZvdXJzOjwvcD4N
Cg0KPHVsPg0KPGxpPnRhcmdldCZsdDthcHBsaWNhdGlvbiZndDsuV0FSPC9saT4NCjxsaT50
YXJnZXQmbHQ7YXBwbGljYXRpb24mZ3Q7LW9yaWdpbmFsLndhcjwvbGk+DQo8L3VsPg0KDQo8
cD5UaGUgZmlyc3Qgb25lIGNvbnRhaW5zIGFsc28gdGhlIGphcnMgZm9yIHJ1bm5pbmcgd2l0
aCBlbWJlZGVkIGFwcGxpY2F0aW9uIHNlcnZlciBidXQgeW91IGNhbiBzYWZlbHkgZGVwbG95
IGl0LjwvcD4NCg0KPHA+PHN0cm9uZz5SZW1hcms6PC9zdHJvbmc+IEpldHR5IHdhcyBydW5u
aW5nIGFzIDxjb2RlPnJvb3Q8L2NvZGU+IHNpbmNlIG5vIDxjb2RlPkpFVFRZX1VTRVI8L2Nv
ZGU+IGVudmlyb25tZW50IHZhcmlhYmxlIHdhcyBkZWZpbmVkIHlldC48L3A+DQoNCjxwPkRl
cGxveW1lbnQgbG9va2VkIHN1Y2Nlc3NmdWxsIChubyBlcnJvcnMgaW4gbG9ncykgYnV0IG5v
dGhpbmcgaGFwcGVuZWQuIDwvcD4NCg0KPHA+QWZ0ZXIgc29tZSB0cmlhbC9lcnJvciBhdHRl
bXB0cyBpdCB3YXMgcXVpdGUgb2J2aW91cyB0aGF0IDxjb2RlPmNvbmZpZ3VyZTwvY29kZT4g
bWV0aG9kIHdhcyBub3QgY2FsbGVkIHNvIG5vIGNoYW5jZSBmb3IgU3ByaW5nIEJvb3QgaW5p
dGlhbGl6YXRpb24gcHJvY2VzcyB0byBiZSB0cmlnZ2VyZWQuIE5vdGhpbmcgaW4gb2ZmaWNp
YWwgU3ByaW5nIEJvb3QgcmVmZXJlbmNlIHdhcyBtZW50aW9uaW5nIGFib3V0IGVuYWJsaW5n
IDxjb2RlPmFub3RhdGlvbnM8L2NvZGU+IG1vZHVsZSBvbiBKZXR0eSwgb2YgY291cnNlIHdl
IGhhdmUgaGFkIHRvIGJlIG1vcmUgYXR0ZW50aXZlIGluIG91ciBraW5kZXJnYXJ0ZW4gSmF2
YSBjb3Vyc2VzJiM4MjMwOyhJIGFtIGEgbmV3YmllIGluIFNwcmluZy9KZXR0eSB3b3JsZCwg
d2h5IG5vdCB0byBmaW5kIG91dCB0aGluZ3MgdGhlIGhhcmQgd2F5ISk8L3A+DQoNCjxoMiBp
ZD0ic3RlcDIiPlN0ZXAgMjwvaDI+DQoNCjxwPkFmdGVyIGVuYWJsaW5nIHRoZSA8Y29kZT5h
bm5vdGF0aW9uPC9jb2RlPiBtb2R1bGUgYW5vdGhlciBzdXJwcmlzZSYjODIzMDthcHBsaWNh
dGlvbiBzdGlsbCBub3Qgd29ya2luZzogSSYjODIxNzt2ZSBnb3QgKGJ5IGFuYWx5c2luZyBs
b2dzKSBhbiBlcnJvciBzaW1pbGFyIHRvIHRoZSBvbmUgZGVzY3JpYmVkIGhlcmU6IDxhIGhy
ZWY9Imh0dHBzOi8vc3RhY2tvdmVyZmxvdy5jb20vcXVlc3Rpb25zLzQxOTk1MDI5L2plcnNl
eS1zcHJpbmczLTItMjUtMS1wcm9kdWNlcy1mYWlsZWQtc3RhcnR1cC1vZi1jb250ZXh0LWVy
cm9yLWluLWpldHR5LTktMyI+aHR0cHM6Ly9zdGFja292ZXJmbG93LmNvbS9xdWVzdGlvbnMv
NDE5OTUwMjkvamVyc2V5LXNwcmluZzMmIzgyMTE7MiYjODIxMTsyNSYjODIxMTsxLXByb2R1
Y2VzLWZhaWxlZC1zdGFydHVwLW9mLWNvbnRleHQtZXJyb3ItaW4tamV0dHkmIzgyMTE7OSYj
ODIxMTszPC9hPi4gUmVtb3ZpbmcgSmVyc2V5IGRlcGVuZGVuY3kgaW4gTWF2ZW4gcG9tLnht
bCBzb2x2ZWQgdGhlIHByb2JsZW0gKHNvbWVob3csIGF0IHNvbWUgcG9pbnQgaW4gZnV0dXJl
IEkgcGxhbiB0byBoYXZlIHNvbWUgUkVTVCBBUEkgZXhwb3NlZCB0byB0aGUgd29ybGQsIGZp
bmdlcnMgY3Jvc3MhKTwvcD4NCg0KPGgyIGlkPSJzdGVwMyI+U3RlcCAzPC9oMj4NCg0KPHA+
V2l0aG91dCByZWFsbHkgdW5kZXJzdGFuZGluZyB3aGF0IEkgd2FzIGRvaW5nIGF0IHRoZSBw
aGFzZSBvZiBnZW5lcmF0aWcgdGhlIHByb2plY3Qgc2NoZWxldG9uIHRocm91Z2ggPGEgaHJl
Zj0iaHR0cHM6Ly9zdGFydC5zcHJpbmcuaW8vIj5odHRwczovL3N0YXJ0LnNwcmluZy5pby88
L2E+IEkgYWRkZWQgPGNvZGU+ZGV2dG9vbHM8L2NvZGU+IHRvIHRoZSBwcm9qZWN0LiBXZWxs
LCB0cnlpbmcgdG8gZGVwbG95IHRvIEpldHR5IHRoZSBXQVIgSSYjODIxNzt2ZSBnb3QgYW5v
dGhlciBlcnJvciwgc29tZXRoaW5nIHdpdGggbm90IGZpbmRpbmcgYW4gZW50cnkgcG9pbnQg
Zm9yIHJlc3RhcnRpbmcgdGhlIGFwcGxpY2F0aW9uLiBUaWxsIHJlc3RhcnRpbmcgSSBuZWVk
ZSBhIGZpcnN0IHByb3BlciBzdGFydCBzbyYjODIzMDtnb29kIGJ5IDxjb2RlPmRldnRvb2xz
PC9jb2RlPi48L3A+DQoNCjxoMiBpZD0ic3RlcDQiPlN0ZXAgNDwvaDI+DQoNCjxwPk5vdyBh
ZGRlZCA8Y29kZT5KRVRUWV9VU0VSPWpldHR5PC9jb2RlPiB0byA8Y29kZT4vZW52L2RlZmF1
bHRzL2pldHR5PC9jb2RlPjwvcD4NCg0KPHA+QW5vdGhlciBzdXJwcmlzZSYjODIzMDtKZXR0
eSB3b3VsZCBub3Qgc3RhcnQuIEFmdGVyIGFub3RoZXIgbGF0ZSBuaWdodCBob3VycyBvZiB0
cmlhbCBhbmQgZXJyb3IgYXR0ZW1wdHMgSSBub3RpY2VkIHRoYXQgaWYgSSBjaGFuZ2Ugb3du
ZXJzaGlwIG9mIFxydW5camV0dHkgdG8gdXNlciBqZXR0eSB0aGUgc2VydmVyIHN0YXJ0cyBh
bmQgSSBjYW4gZGVwbG95IHRoZSBXQVIuIFRoZSBwcm9ibGVtIHdhcyB0aGF0IGFmdGVyIHJl
c3RyYXRpbmcgdGhlIHNlcnZlciB0aGUgcGVybWlzc2lvbnMgb24gL3J1biBkaXJlY3Rvcnkg
d2VyZSBsb3N0Lg0KT2ssIHRpbWUgdG8gYW5hbHl6ZSA8Y29kZT4vZXRjL2luaXQuZC9qZXR0
eTwvY29kZT4uIFRoZSBzaG9ydCBzdG9yeSAoYWxzbyBkZXNjcmliZWQgaGVyZTogPGEgaHJl
Zj0iaHR0cHM6Ly9zdGFja292ZXJmbG93LmNvbS9xdWVzdGlvbnMvMTc5OTk3MjkvamV0dHkt
YmFzaC1zY3JpcHQtd29ya3Mtb25seS13aXRoLXJvb3QtdXNlciI+aHR0cHM6Ly9zdGFja292
ZXJmbG93LmNvbS9xdWVzdGlvbnMvMTc5OTk3MjkvamV0dHktYmFzaC1zY3JpcHQtd29ya3Mt
b25seS13aXRoLXJvb3QtdXNlcjwvYT4pOjwvcD4NCg0KPHA+VGhlIGV2aWwgaGlkZXMgYmVo
aW5kIHRoZSBmb2xsb3dpbmcgc25pcHBldDoNCjxjb2RlPg0KaWYgWyAteiAmcXVvdDskSkVU
VFlfUlVOJnF1b3Q7IF0NCnRoZW4NCiAgSkVUVFlfUlVOPSQoZmluZERpcmVjdG9yeSAtdyAv
dmFyL3J1biAvdXNyL3Zhci9ydW4gJEpFVFRZX0JBU0UgL3RtcCkvamV0dHkNCiAgWyAtZCAm
cXVvdDskSkVUVFlfUlVOJnF1b3Q7IF0gfHwgbWtkaXIgJEpFVFRZX1JVTg0KZmkNCjwvY29k
ZT4NCklmIG5vIDxjb2RlPkpFVFRZX1JVTjwvY29kZT4gZW52aXJvbm1lbnQgdmFyaWFibGUg
aXMgZGVmaW5lZCB0aGUgc2NyaXB0IGF0dGVtcHRzIHRvIGJ1aWxkIGEgYmFzZSBvbmUgYW5k
IGFwcGVuZHMgPGNvZGU+amV0dHk8L2NvZGU+IHRvIGl0OyBzbyB0aGUgZGlyZWN0b3J5IGlz
IGNyZWF0ZWQgKHN1cnByaXNlKSB1bmRlciA8Y29kZT5yb290PC9jb2RlPiBhY2NvdW50IGFu
ZCBsYXRlciBhdHRlbXB0IHRvIHN0YXJ0IGpldHR5ICg8Y29kZT5zdGFydC1zdG9wLWRhZW1v
biAtUyAtcCZxdW90OyRKRVRUWV9QSUQmcXVvdDsgJENIX1VTRVIgLWQmcXVvdDskSkVUVFlf
QkFTRSZxdW90OyAtYiAtbSAtYSAmcXVvdDskSkFWQSZxdW90OyAtLSAmcXVvdDske1JVTl9B
UkdTW0BdfSZxdW90OyBzdGFydC1sb2ctZmlsZT0mcXVvdDskSkVUVFlfU1RBUlRfTE9HJnF1
b3Q7PC9jb2RlPikgd2l0aCA8Y29kZT5KRVRUWV9TVEFSVF9MT0c9JnF1b3Q7JEpFVFRZX1JV
Ti8kTkFNRS1zdGFydC5sb2cmcXVvdDs8L2NvZGU+IHdpbGwgZmFpbCBzaW5jZSBqZXR0eSB1
c2VyIHdpbGwgbm90IGJlIGFibGUgdG8gY3JlYXRlIHRoZSBsb2cgZmlsZS48L3A+DQoNCjxw
PlRoZSBzb2x1dGlvbiB3YXMgdG8gcHJlLWRlZmluZSB0aGUgPGNvZGU+SkVUVFlfUlVOPC9j
b2RlPiBlbnZpcm9ubWVudCB2YXJpYWJsZSBwb2ludGluZyB0byBhIGRpcmVjdG9yeSB3aGVy
ZSBqZXR0eSBpcyBvd25lci48L3A+DQoNCjxoMSBpZD0iYXBwZW5kaXgiPkFwcGVuZGl4PC9o
MT4NCg0KPGgyIGlkPSJmdWxsamV0dHlpbnN0YWxsYXRpb25zdGVwcyI+RnVsbCBKZXR0eSBp
bnN0YWxsYXRpb24gc3RlcHM8L2gyPg0KDQo8aDMgaWQ9ImphdmE4Ij5KYXZhIDg8L2gzPg0K
DQo8cHJlPjxjb2RlPnN1ZG8gLWkNCjwvY29kZT48L3ByZT4NCg0KPHA+UnVuIG9uZSBieSBv
bmUgdGhlIGZvbGxvd2luZyBjb21tYW5kczt3YXRjaCBvdXQgZm9yIGNhcGl0YWxpemVkIDxj
b2RlPlk8L2NvZGU+IG9uIGFuc3dlcnMuPC9wPg0KDQo8cHJlPjxjb2RlPmFwdCBpbnN0YWxs
IHNvZnR3YXJlLXByb3BlcnRpZXMtY29tbW9uDQphZGQtYXB0LXJlcG9zaXRvcnkgcHBhOndl
YnVwZDh0ZWFtL2phdmENCmFwdC1nZXQgdXBkYXRlDQphcHQtZ2V0IGluc3RhbGwgb3JhY2xl
LWphdmE4LWluc3RhbGxlcg0KDQojYXB0LWdldCBpbnN0YWxsICBvcmFjbGUtamF2YTgtc2V0
LWRlZmF1bHQNCjwvY29kZT48L3ByZT4NCg0KPGgzIGlkPSJkb3dubG9hZGFuZGluc3RhbGxq
ZXR0eSI+RG93bmxvYWQgYW5kIGluc3RhbGwgSmV0dHk8L2gzPg0KDQo8cHJlPjxjb2RlPnN1
ZG8gLWkNCjwvY29kZT48L3ByZT4NCg0KPHByZT48Y29kZT5ta2RpciAtcCAvb3B0L2pldHR5
DQpta2RpciAtcCAvb3B0L3dlYi9qZXR0eWJhc2UNCm1rZGlyIC9vcHQvd2ViL2pldHR5YmFz
ZS9ydW4NCm1rZGlyIC1wIC9vcHQvamV0dHkvdGVtcA0KdXNlcmFkZCAtLXVzZXItZ3JvdXAg
LS1zaGVsbCAvYmluL2ZhbHNlIC0taG9tZS1kaXIgL29wdC9qZXR0eS90ZW1wIGpldHR5DQpj
aG93biAtUiBqZXR0eTpqZXR0eSAvb3B0L2pldHR5DQpjaG93biAtUiBqZXR0eTpqZXR0eSAv
b3B0L3dlYi9qZXR0eWJhc2UNCmNob3duIC1SIGpldHR5OmpldHR5IC9vcHQvamV0dHkvdGVt
cA0KPC9jb2RlPjwvcHJlPg0KDQo8cHJlPjxjb2RlPmNkIC90bXANCndnZXQgaHR0cDovL2Nl
bnRyYWwubWF2ZW4ub3JnL21hdmVuMi9vcmcvZWNsaXBzZS9qZXR0eS9qZXR0eS1kaXN0cmli
dXRpb24vOS40LjYudjIwMTcwNTMxL2pldHR5LWRpc3RyaWJ1dGlvbi05LjQuNi52MjAxNzA1
MzEudGFyLmd6DQpjZCAvb3B0L2pldHR5DQoNCnRhciAtenhmIC90bXAvamV0dHktZGlzdHJp
YnV0aW9uLTkuNC42LnYyMDE3MDUzMS50YXIuZ3ogDQpjZCAvb3B0L2pldHR5L2pldHR5LWRp
c3RyaWJ1dGlvbi05LjQuNi52MjAxNzA1MzENCg0KY3AgYmluL2pldHR5LnNoIC9ldGMvaW5p
dC5kL2pldHR5DQoNCmNkIC9vcHQvd2ViL2pldHR5YmFzZQ0KDQpqYXZhIC1qYXIgL29wdC9q
ZXR0eS9qZXR0eS1kaXN0cmlidXRpb24tOS40LjYudjIwMTcwNTMxL3N0YXJ0LmphciAtLWFk
ZC10by1zdGFydD1kZXBsb3ksaHR0cCxjb25zb2xlLWNhcHR1cmUsYW5ub3RhdGlvbnMNCjwv
Y29kZT48L3ByZT4NCg0KPHByZT48Y29kZT4jY29uZmlndXJlcyBKYXZhIGZvciBkZWJ1Zywg
bXVzdCBiZSByZW1vdmVkIGZyb20gcHJvZHVjdGlvbiBlbnZpcm9ubWVtbnRzDQplY2hvICZx
dW90O0pFVFRZX0hPTUU9L29wdC9qZXR0eS9qZXR0eS1kaXN0cmlidXRpb24tOS40LjYudjIw
MTcwNTMxJnF1b3Q7ICZndDsgL2V0Yy9kZWZhdWx0L2pldHR5DQplY2hvICZxdW90O0pFVFRZ
X0JBU0U9L29wdC93ZWIvamV0dHliYXNlJnF1b3Q7ICZndDsmZ3Q7IC9ldGMvZGVmYXVsdC9q
ZXR0eQ0KZWNobyAmcXVvdDtKRVRUWV9SVU49L29wdC93ZWIvamV0dHliYXNlL3J1biZxdW90
OyAmZ3Q7Jmd0OyAvZXRjL2RlZmF1bHQvamV0dHkNCmVjaG8gJnF1b3Q7VE1QRElSPS9vcHQv
amV0dHkvdGVtcCZxdW90OyAmZ3Q7Jmd0OyAvZXRjL2RlZmF1bHQvamV0dHkNCmVjaG8gJnF1
b3Q7SkFWQV9PUFRJT05TPVwmcXVvdDstWGRlYnVnIC1hZ2VudGxpYjpqZHdwPXRyYW5zcG9y
dD1kdF9zb2NrZXQsYWRkcmVzcz05OTk5LHNlcnZlcj15LHN1c3BlbmQ9blwmcXVvdDsmcXVv
dDsgJmd0OyZndDsgL2V0Yy9kZWZhdWx0L2pldHR5DQplY2hvICZxdW90O0pFVFRZX1VTRVI9
amV0dHkmcXVvdDsgJmd0OyZndDsgL2V0Yy9kZWZhdWx0L2pldHR5DQoNCg0Kc2VydmljZSBq
ZXR0eSBjaGVjaw0KDQp1cGRhdGUtcmMuZCBqZXR0eSBkZWZhdWx0cw0KDQpzZXJ2aWNlIGpl
dHR5IHN0YXJ0DQoNCnNlcnZpY2UgamV0dHkgc3RhdHVzDQoNCiMgaWYgc2VydmljZSBub3Qg
c3RhcnRpbmcgY2hlY2sgYWxsIGludm9sdmVkIGRpcmVjdG9yaWVzIHRoYXQgamV0dHkgdXNl
ciBpcyBvd25lcg0KDQojIGp1c3QgZm9yIGRldmVsb3BtZW50LCBpbiBvcmRlciB0byBiZSBh
YmxlIHRvIGNvcHkgZmlsZXMgdGhyb3VnaCBXaW5TQ1A6DQpjaG1vZCBvK3cgL29wdC93ZWIv
amV0dHliYXNlL3dlYmFwcHMNCjwvY29kZT48L3ByZT4NCg0KPGgyIGlkPSJob3dzcHJpbmdi
b290Ym9vdHNyYXBzaG91bGR3b3JrIj5Ib3cgU3ByaW5nIEJvb3QgQm9vdHNyYXAgU2hvdWxk
IFdvcms8L2gyPg0KDQo8cD5TZWUgYWxzbyA8YSBocmVmPSJodHRwOi8vcGlvdHJub3dpY2tp
LmNvbS8yMDExLzAzL3VzaW5nLXNlcnZsZXRzLTMtMC1zZXJ2bGV0Y29udGFpbmVyaW5pdGlh
bGl6ZXIvIj5odHRwOi8vcGlvdHJub3dpY2tpLmNvbS8yMDExLzAzL3VzaW5nLXNlcnZsZXRz
JiM4MjExOzMmIzgyMTE7MC1zZXJ2bGV0Y29udGFpbmVyaW5pdGlhbGl6ZXIvPC9hPjwvcD4N
Cg0KPHA+VGhlICZndDs9My4wIFNlcnZsZXQgQ29udGFpbmVyIChKZXR0eSBpbiBvdXIgY2Fz
ZSkgd2lsbCB0cnkgKGlmIGFubm90YXRpb25zIG1vZHVsZSBpcyBhY3RpdmF0ZWQhKSB0byBm
aW5kIGluIGNsYXNzIHBhdGggYWxsIGltcGxlbWVudGF0aW9ucyBvZiA8Y29kZT5qYXZheC5z
ZXJ2bGV0LlNlcnZsZXRDb250YWluZXJJbml0aWFsaXplcjwvY29kZT4gYW5kIHdpbGwgY2Fs
bCBpdHMgPGNvZGU+b25TdGFydHVwPC9jb2RlPiBtZXRob2Qgd2l0aCBhIDxjb2RlPlNldDwv
Y29kZT4gb2YgYWxsIGltcGxlbWVudGF0aW9ucyBvZiB0aGUgPGNvZGU+QEhhbmRsZXNUeXBl
czwvY29kZT4gbWVudGlvbmVkIGNsYXNzOjwvcD4NCg0KPHByZT48Y29kZT5wYWNrYWdlIG9y
Zy5zcHJpbmdmcmFtZXdvcmsud2ViOw0KLi4uDQoNCkBIYW5kbGVzVHlwZXMoV2ViQXBwbGlj
YXRpb25Jbml0aWFsaXplci5jbGFzcykNCnB1YmxpYyBjbGFzcyBTcHJpbmdTZXJ2bGV0Q29u
dGFpbmVySW5pdGlhbGl6ZXIgaW1wbGVtZW50cyBTZXJ2bGV0Q29udGFpbmVySW5pdGlhbGl6
ZXIgew0KDQogICAgLyoqDQogICAgICogRGVsZWdhdGUgdGhlIHtAY29kZSBTZXJ2bGV0Q29u
dGV4dH0gdG8gYW55IHtAbGluayBXZWJBcHBsaWNhdGlvbkluaXRpYWxpemVyfQ0KICAgICAq
IGltcGxlbWVudGF0aW9ucyBwcmVzZW50IG9uIHRoZSBhcHBsaWNhdGlvbiBjbGFzc3BhdGgu
DQogICAgICogJmx0O3AmZ3Q7QmVjYXVzZSB0aGlzIGNsYXNzIGRlY2xhcmVzIEB7QGNvZGUg
SGFuZGxlc1R5cGVzKFdlYkFwcGxpY2F0aW9uSW5pdGlhbGl6ZXIuY2xhc3MpfSwNCiAgICAg
KiBTZXJ2bGV0IDMuMCsgY29udGFpbmVycyB3aWxsIGF1dG9tYXRpY2FsbHkgc2NhbiB0aGUg
Y2xhc3NwYXRoIGZvciBpbXBsZW1lbnRhdGlvbnMNCiAgICAgKiBvZiBTcHJpbmcncyB7QGNv
ZGUgV2ViQXBwbGljYXRpb25Jbml0aWFsaXplcn0gaW50ZXJmYWNlIGFuZCBwcm92aWRlIHRo
ZSBzZXQgb2YgYWxsDQogICAgICogc3VjaCB0eXBlcyB0byB0aGUge0Bjb2RlIHdlYkFwcElu
aXRpYWxpemVyQ2xhc3Nlc30gcGFyYW1ldGVyIG9mIHRoaXMgbWV0aG9kLg0KICAgICAqICZs
dDtwJmd0O0lmIG5vIHtAY29kZSBXZWJBcHBsaWNhdGlvbkluaXRpYWxpemVyfSBpbXBsZW1l
bnRhdGlvbnMgYXJlIGZvdW5kIG9uIHRoZSBjbGFzc3BhdGgsDQogICAgICogdGhpcyBtZXRo
b2QgaXMgZWZmZWN0aXZlbHkgYSBuby1vcC4gQW4gSU5GTy1sZXZlbCBsb2cgbWVzc2FnZSB3
aWxsIGJlIGlzc3VlZCBub3RpZnlpbmcNCiAgICAgKiB0aGUgdXNlciB0aGF0IHRoZSB7QGNv
ZGUgU2VydmxldENvbnRhaW5lckluaXRpYWxpemVyfSBoYXMgaW5kZWVkIGJlZW4gaW52b2tl
ZCBidXQgdGhhdA0KICAgICAqIG5vIHtAY29kZSBXZWJBcHBsaWNhdGlvbkluaXRpYWxpemVy
fSBpbXBsZW1lbnRhdGlvbnMgd2VyZSBmb3VuZC4NCiAgICAgKiAmbHQ7cCZndDtBc3N1bWlu
ZyB0aGF0IG9uZSBvciBtb3JlIHtAY29kZSBXZWJBcHBsaWNhdGlvbkluaXRpYWxpemVyfSB0
eXBlcyBhcmUgZGV0ZWN0ZWQsDQogICAgICogdGhleSB3aWxsIGJlIGluc3RhbnRpYXRlZCAo
YW5kICZsdDtlbSZndDtzb3J0ZWQmbHQ7L2VtJmd0OyBpZiB0aGUgQHtAbGluaw0KICAgICAq
IG9yZy5zcHJpbmdmcmFtZXdvcmsuY29yZS5hbm5vdGF0aW9uLk9yZGVyIEBPcmRlcn0gYW5u
b3RhdGlvbiBpcyBwcmVzZW50IG9yDQogICAgICogdGhlIHtAbGluayBvcmcuc3ByaW5nZnJh
bWV3b3JrLmNvcmUuT3JkZXJlZCBPcmRlcmVkfSBpbnRlcmZhY2UgaGFzIGJlZW4NCiAgICAg
KiBpbXBsZW1lbnRlZCkuIFRoZW4gdGhlIHtAbGluayBXZWJBcHBsaWNhdGlvbkluaXRpYWxp
emVyI29uU3RhcnR1cChTZXJ2bGV0Q29udGV4dCl9DQogICAgICogbWV0aG9kIHdpbGwgYmUg
aW52b2tlZCBvbiBlYWNoIGluc3RhbmNlLCBkZWxlZ2F0aW5nIHRoZSB7QGNvZGUgU2Vydmxl
dENvbnRleHR9IHN1Y2gNCiAgICAgKiB0aGF0IGVhY2ggaW5zdGFuY2UgbWF5IHJlZ2lzdGVy
IGFuZCBjb25maWd1cmUgc2VydmxldHMgc3VjaCBhcyBTcHJpbmcncw0KICAgICAqIHtAY29k
ZSBEaXNwYXRjaGVyU2VydmxldH0sIGxpc3RlbmVycyBzdWNoIGFzIFNwcmluZydzIHtAY29k
ZSBDb250ZXh0TG9hZGVyTGlzdGVuZXJ9LA0KICAgICAqIG9yIGFueSBvdGhlciBTZXJ2bGV0
IEFQSSBjb21wb25lbnRyeSBzdWNoIGFzIGZpbHRlcnMuDQogICAgICogQHBhcmFtIHdlYkFw
cEluaXRpYWxpemVyQ2xhc3NlcyBhbGwgaW1wbGVtZW50YXRpb25zIG9mDQogICAgICoge0Bs
aW5rIFdlYkFwcGxpY2F0aW9uSW5pdGlhbGl6ZXJ9IGZvdW5kIG9uIHRoZSBhcHBsaWNhdGlv
biBjbGFzc3BhdGgNCiAgICAgKiBAcGFyYW0gc2VydmxldENvbnRleHQgdGhlIHNlcnZsZXQg
Y29udGV4dCB0byBiZSBpbml0aWFsaXplZA0KICAgICAqIEBzZWUgV2ViQXBwbGljYXRpb25J
bml0aWFsaXplciNvblN0YXJ0dXAoU2VydmxldENvbnRleHQpDQogICAgICogQHNlZSBBbm5v
dGF0aW9uQXdhcmVPcmRlckNvbXBhcmF0b3INCiAgICAgKi8NCiAgICBAT3ZlcnJpZGUNCiAg
ICBwdWJsaWMgdm9pZCBvblN0YXJ0dXAoU2V0Jmx0O0NsYXNzJmx0Oz8mZ3Q7Jmd0OyB3ZWJB
cHBJbml0aWFsaXplckNsYXNzZXMsIFNlcnZsZXRDb250ZXh0IHNlcnZsZXRDb250ZXh0KQ0K
ICAgICAgICAgICAgdGhyb3dzIFNlcnZsZXRFeGNlcHRpb24gew0KDQogICAgICAgIExpc3Qm
bHQ7V2ViQXBwbGljYXRpb25Jbml0aWFsaXplciZndDsgaW5pdGlhbGl6ZXJzID0gbmV3IExp
bmtlZExpc3QmbHQ7V2ViQXBwbGljYXRpb25Jbml0aWFsaXplciZndDsoKTsNCg0KICAgICAg
ICBpZiAod2ViQXBwSW5pdGlhbGl6ZXJDbGFzc2VzICE9IG51bGwpIHsNCiAgICAgICAgICAg
IGZvciAoQ2xhc3MmbHQ7PyZndDsgd2FpQ2xhc3MgOiB3ZWJBcHBJbml0aWFsaXplckNsYXNz
ZXMpIHsNCiAgICAgICAgICAgICAgICAvLyBCZSBkZWZlbnNpdmU6IFNvbWUgc2VydmxldCBj
b250YWluZXJzIHByb3ZpZGUgdXMgd2l0aCBpbnZhbGlkIGNsYXNzZXMsDQogICAgICAgICAg
ICAgICAgLy8gbm8gbWF0dGVyIHdoYXQgQEhhbmRsZXNUeXBlcyBzYXlzLi4uDQouLi4NCg0K
PC9jb2RlPjwvcHJlPg0K