My LiFe StOrY: 3월 2017

아파치2.4 와 톰켓7.0 을 이용한 로드벨런싱, 세션 클러스터 환경 구성

1) 테스트 환경
- os : ubuntu 16.04 64bit
- java : 1.7.0_80
- apache : 2.4.25
- tomcat : 7.0.76

2) java 설치
# tar zxvf jdk-7u80-linux-x64.tar.gz -C /usr/local/
# ln -s /usr/local/jdk1.7.0_80/bin/java /usr/bin/java

3) Apach2.4 설치를 위한 APR, APR-util 및 의존 패키지 설치

* 아파치 2.4 버젼 부터는 APR 과 APR-util 을 따로 컴파일 하여 설치 하여야 한다고 한다.
http://apr.apache.org/download.cgi 에서 다운로드

3-1) APR 다운로드 및 컴파일, 설치
# wget http://apache.mirror.cdnetworks.com/apr/apr-1.5.2.tar.gz
# tar zxvf apr-1.5.2.tar.gz
# cd apr-1.5.2/
# ./configure
# make
# make install

3-2) APR-util 다운로드 및 컴파일, 설치
# wget http://apache.mirror.cdnetworks.com/apr/apr-util-1.5.4.tar.gz
# tar zxvf apr-util-1.5.4.tar.gz
# cd apr-util-1.5.4
# ./configure --with-apr=/usr/local/apr
# make
# make install

3-3) 의존 패키지 설치
# apt-get install libpcre3-dev
* RPM 기반의 CentOS7 의 경우 yum을 이용하여 pcre-devel 패키지를 설치 한다.

4) Apach2.4 설치(컴파일) 및 구동 테스트
# wget http://apache.mirror.cdnetworks.com/httpd/httpd-2.4.25.tar.gz
# tar zxvf httpd-2.4.25.tar.gz
# cd httpd-2.4.25
# ./configure --prefix=/usr/local/apache-2.4.25
# make
# make install

# /usr/local/apache-2.4.25/bin/apachectl start
* 브라우저를 이용하여 http://localhost 에 접속하여 본다.

# /usr/local/apache-2.4.25/bin/apachectl stop

5) Apach2.4 설정 파일 수정
# cp /usr/local/apache-2.4.25/conf/httpd.conf /usr/local/apache-2.4.25/conf/httpd.conf.ori
# vi /usr/local/apache-2.4.25/conf/httpd.conf

* 아래의 내용들을 찾아 주석 해제 한다.

LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
LoadModule status_module modules/mod_status.so
LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so
LoadModule lbmethod_bytraffic_module modules/mod_lbmethod_bytraffic.so
LoadModule lbmethod_bybusyness_module modules/mod_lbmethod_bybusyness.so
LoadModule lbmethod_heartbeat_module modules/mod_lbmethod_heartbeat.so

Include conf/extra/httpd-vhosts.conf

# cp /usr/local/apache-2.4.25/conf/extra/httpd-vhosts.conf /usr/local/apache-2.4.25/conf/extra/httpd-vhosts.conf.ori
# vi /usr/local/apache-2.4.25/conf/extra/httpd-vhosts.conf
* httpd-vhosts.conf 기존 내용 제거 후 아래의 내용을 수정 하여 추가 한다.

<VirtualHost *:80>
ServerAdmin admin@test.co.kr
ServerName test.co.kr

ProxyRequests Off
ProxyPass / balancer://mycluster/
ProxyPassReverse / balancer://mycluster

<Proxy balancer://mycluster stickysession=JSESSIONID|jsessionid scolonpathdelim=On>
BalancerMember http://127.0.0.1:8080 route=1
BalancerMember http://127.0.0.1:8081 route=2
</Proxy>
</VirtualHost>

6) Apach2.4 서비스 등록
# vi /etc/init.d/apache

------------------- 파일 내용 // -------------------------
#!/bin/bash

### BEGIN INIT INFO
# Provides: apache
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: auto start apache web server
# Description: Start Web Server
### END INIT INFO

export APACHE_HOME="/usr/local/apache-2.4.25/"

case $1 in
start)
sh $APACHE_HOME/bin/apachectl -k start
;;
stop)
sh $APACHE_HOME/bin/apachectl -k stop
;;
restart)
sh $APACHE_HOME/bin/apachectl -k stop
sh $APACHE_HOME/bin/apachectl -k start
;;
esac
exit 0
------------------- // 파일 내용 -------------------------

# chmod 755 /etc/init.d/apache
# update-rc.d apache defaults
# chkconfig --add apache
# chkconfig -list | grep -i apache
apache 0:off 1:off 2:on 3:on 4:on 5:on 6:off
# /etc/init.d/apache start
httpd (pid 1771) already running

# ps -ef | grep -i apache
root 670 1 0 15:05 ? 00:00:00 /usr/local/apache-2.4.25/bin/httpd -k start
daemon 671 670 0 15:05 ? 00:00:00 /usr/local/apache-2.4.25/bin/httpd -k start
daemon 672 670 0 15:05 ? 00:00:00 /usr/local/apache-2.4.25/bin/httpd -k start
daemon 674 670 0 15:05 ? 00:00:00 /usr/local/apache-2.4.25/bin/httpd -k start

7) tomcat 사용자 생성
# useradd -m tomcat
# echo 'tomcat:p@ssw0rd' | chpasswd

* 배포판에 따라 chpasswd 명령이 없는 경우 passwd tomcat 명령으로 tomcat 사용자의 비밀번호를 변경 한다.

8) Tomcat 7.0 다운로드 및 환경설정
# su - tomcat
$ cd
$ pwd
/home/tomcat
$ wget http://apache.mirror.cdnetworks.com/tomcat/tomcat-7/v7.0.76/bin/apache-tomcat-7.0.76.tar.gz

8-1)Tomcat 1호기, 2호기 압축 해제 및 시작, 종료 명령 변경
$ tar zxvf apache-tomcat-7.0.76.tar.gz -C apache-tomcat-7.0.76_1
$ tar zxvf apache-tomcat-7.0.76.tar.gz -C apache-tomcat-7.0.76_2

$ mv ~/apache-tomcat-7.0.76_1/bin/startup.sh ~/apache-tomcat-7.0.76_1/bin/startup1.sh
$ mv ~/apache-tomcat-7.0.76_1/bin/shutdown.sh ~/apache-tomcat-7.0.76_1/bin/shutdown1.sh

$ mv ~/apache-tomcat-7.0.76_2/bin/startup.sh ~/apache-tomcat-7.0.76_2/bin/startup2.sh
$ mv ~/apache-tomcat-7.0.76_2/bin/shutdown.sh ~/apache-tomcat-7.0.76_2/bin/shutdown2.sh

8-2) Tomcat 1호기 의 catalina.sh 파일 내용에 추가(절대 경로 입력)
$ cp ~/apache-tomcat-7.0.76_1/bin/catalina.sh ~/apache-tomcat-7.0.76_1/bin/catalina.sh.ori
$ vi ~/apache-tomcat-7.0.76_1/bin/catalina.sh
...중략...
export CATALINA_HOME=/home/tomcat/apache-tomcat-7.0.76_1
export CATALINA_BASE=/home/tomcat/apache-tomcat-7.0.76_1
CATALINA_PID=/home/tomcat/apache-tomcat-7.0.76_1/bin/tomcat.pid
...중략...

8-3) Tomcat 2호기 의 catalina.sh 파일 내용에 추가(절대 경로 입력)
$ cp ~/apache-tomcat-7.0.76_2/bin/catalina.sh ~/apache-tomcat-7.0.76_2/bin/catalina.sh.ori
$ vi ~/apache-tomcat-7.0.76_2/bin/catalina.sh
...중략...
export CATALINA_HOME=/home/tomcat/apache-tomcat-7.0.76_2
export CATALINA_BASE=/home/tomcat/apache-tomcat-7.0.76_2
CATALINA_PID=/home/tomcat/apache-tomcat-7.0.76_2/bin/tomcat.pid
...중략...

8-4) Tomcat 1,2 호기의 시작, 종료 명령을 PATH 에 등록
$ vi ~/.bashrc
...중략...
export JAVA_HOME="/usr/local/jdk1.7.0_80/"
export CATALINA_HOME1=/home/tomcat/apache-tomcat-7.0.76_1
export CATALINA_HOME2=/home/tomcat/apache-tomcat-7.0.76_2
export PATH=$PATH:$JAVA_HOME/bin:$CATALINA_HOME1/bin:$CATALINA_HOME2/bin
...중략...

$ source ~/.bashrc
$ java -version
java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)

8-5) Tomcat 1호기의 server.xml 파일 수정
$ cp ~/apache-tomcat-7.0.76_1/conf/server.xml ~/apache-tomcat-7.0.76_1/conf/server.xml.ori
$ vi ~/apache-tomcat-7.0.76_1/conf/server.xml

<Server port="8005" shutdown="SHUTDOWN">
... 중략 ...
<Connector port="8080"
protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443"
disableUploadTimeout="true"
enableLookups="false"
URIEncoding="UTF-8" />
... 중략 ...
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
... 중략 ...
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">
... 중략 ...

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="5">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>

<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>

<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>

<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif|.*\.js|.*\.jpeg|.*\.jpg|.*\.png|.*\.htm|.*\.html|.*\.css|.*\.txt"/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false"/>

<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" />
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener" />
</Cluster>

8-6) Tomcat 2호기의 server.xml 파일 수정
$ cp ~/apache-tomcat-7.0.76_2/conf/server.xml ~/apache-tomcat-7.0.76_2/conf/server.xml.ori
$ vi ~/apache-tomcat-7.0.76_2/conf/server.xml

<Server port="8006" shutdown="SHUTDOWN">
... 중략 ...
<Connector port="8081"
protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443"
disableUploadTimeout="true"
enableLookups="false"
URIEncoding="UTF-8" />
... 중략 ...
<Connector port="8010" protocol="AJP/1.3" redirectPort="8443" />
... 중략 ...
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2">
... 중략 ...

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="5">

<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>

<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4001"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>

<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>

<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif|.*\.js|.*\.jpeg|.*\.jpg|.*\.png|.*\.htm|.*\.html|.*\.css|.*\.txt"/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false"/>

<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" />
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener" />
</Cluster>

8-7) Tomcat 1,2 호기 시작 테스트
$ startup1.sh
Using CATALINA_BASE: /home/tomcat/apache-tomcat-7.0.76_1
Using CATALINA_HOME: /home/tomcat/apache-tomcat-7.0.76_1
Using CATALINA_TMPDIR: /home/tomcat/apache-tomcat-7.0.76_1/temp
Using JRE_HOME: /usr/local/jdk1.7.0_80/
Using CLASSPATH: /home/tomcat/apache-tomcat-7.0.76_1/bin/bootstrap.jar:/home/tomcat/apache-tomcat-7.0.76_1/bin/tomcat-juli.jar
Tomcat started.

$ startup2.sh
Using CATALINA_BASE: /home/tomcat/apache-tomcat-7.0.76_2
Using CATALINA_HOME: /home/tomcat/apache-tomcat-7.0.76_2
Using CATALINA_TMPDIR: /home/tomcat/apache-tomcat-7.0.76_2/temp
Using JRE_HOME: /usr/local/jdk1.7.0_80/
Using CLASSPATH: /home/tomcat/apache-tomcat-7.0.76_2/bin/bootstrap.jar:/home/tomcat/apache-tomcat-7.0.76_2/bin/tomcat-juli.jar
Tomcat started.

* 브라우저를 이용하여 http://192.168.0.1:8080, http://192.168.0.1:8081 에 접속하여 본다.

8-8) Tomcat 1,2 호기 종료 테스트
$ shutdown1.sh
Using CATALINA_BASE: /home/tomcat/apache-tomcat-7.0.76_1
Using CATALINA_HOME: /home/tomcat/apache-tomcat-7.0.76_1
Using CATALINA_TMPDIR: /home/tomcat/apache-tomcat-7.0.76_1/temp
Using JRE_HOME: /usr/local/jdk1.7.0_80/
Using CLASSPATH: /home/tomcat/apache-tomcat-7.0.76_1/bin/bootstrap.jar:/home/tomcat/apache-tomcat-7.0.76_1/bin/tomcat-juli.jar

$ shutdown2.sh
Using CATALINA_BASE: /home/tomcat/apache-tomcat-7.0.76_2
Using CATALINA_HOME: /home/tomcat/apache-tomcat-7.0.76_2
Using CATALINA_TMPDIR: /home/tomcat/apache-tomcat-7.0.76_2/temp
Using JRE_HOME: /usr/local/jdk1.7.0_80/
Using CLASSPATH: /home/tomcat/apache-tomcat-7.0.76_2/bin/bootstrap.jar:/home/tomcat/apache-tomcat-7.0.76_2/bin/tomcat-juli.jar

8-9) Tomcat 메니져 사용을 위한 설정 부분(필수 아님)

* 이 문서에서는 나중에 jenkins 사용을 위해 설정 하였다.

$ cp ~/apache-tomcat-7.0.76_1/conf/tomcat-users.xml ~/apache-tomcat-7.0.76_1/conf/tomcat-users.xml.ori
$ cp ~/apache-tomcat-7.0.76_2/conf/tomcat-users.xml ~/apache-tomcat-7.0.76_2/conf/tomcat-users.xml.ori
$ vi ~/apache-tomcat-7.0.76_1/conf/tomcat-users.xml
...중략...

<role rolename="manager-gui"/>
<role rolename="manager-script"/>
<role rolename="manager-status"/>
<role rolename="manager-gui"/>
<user username="tomcat" password="p@ssW0rd" roles="manager-gui, manager-script, manager-status, manager-gui"/>

$ vi ~/apache-tomcat-7.0.76_2/conf/tomcat-users.xml
...중략...

<role rolename="manager-gui"/>
<role rolename="manager-script"/>
<role rolename="manager-status"/>
<role rolename="manager-gui"/>
<user username="tomcat" password="p@ssW0rd" roles="manager-gui, manager-script, manager-status, manager-gui"/>

9) Tomcat 1,2 호기 서비스 등록
# vi /etc/init.d/tomcat
------------------- 파일 내용 // -------------------------
#!/bin/bash

### BEGIN INIT INFO
# Provides: Tomcat1
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Auto Start Tomcat1 Server
# Description: Start Web Application Server
### END INIT INFO

export CATALINA_HOME="/home/jhkim1981/tomcat/apache-tomcat-7.0.76_1"

case $1 in
start)
su - tomcat -c $CATALINA_HOME/bin/startup1.sh
;;
stop)
su - tomcat -c $CATALINA_HOME/bin/shutdown1.sh
;;
restart)
su - tomcat -c $CATALINA_HOME/bin/shutdown1.sh
su - tomcat -c $CATALINA_HOME/bin/startup1.sh
;;
esac
exit 0
------------------- // 파일 내용 -------------------------

# vi /etc/init.d/tomcat2
------------------- 파일 내용 // -------------------------
#!/bin/bash

### BEGIN INIT INFO
# Provides: Tomcat2
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Auto Start Tomcat2 Server
# Description: Start Web Application Server
### END INIT INFO

export CATALINA_HOME="/home/jhkim1981/tomcat/apache-tomcat-7.0.76_2"

case $1 in
start)
su - tomcat -c $CATALINA_HOME/bin/startup1.sh
;;
stop)
su - tomcat -c $CATALINA_HOME/bin/shutdown1.sh
;;
restart)
su - tomcat -c $CATALINA_HOME/bin/shutdown1.sh
su - tomcat -c $CATALINA_HOME/bin/startup1.sh
;;
esac
exit 0
------------------- // 파일 내용 -------------------------

# chmod 755 /etc/init.d/tomcat1
# chmod 755 /etc/init.d/tomcat2

# update-rc.d tomcat1 defaults
# update-rc.d tomcat2 defaults

# chkconfig --add tomcat1
# chkconfig --add tomcat2

# chkconfig -list | grep -i tomcat
tomcat1 0:off 1:off 2:on 3:on 4:on 5:on 6:off
tomcat2 0:off 1:off 2:on 3:on 4:on 5:on 6:off

# /etc/init.d/tomcat1 start
Using CATALINA_BASE: /home/tomcat/apache-tomcat-7.0.76_1
Using CATALINA_HOME: /home/tomcat/apache-tomcat-7.0.76_1
Using CATALINA_TMPDIR: /home/tomcat/apache-tomcat-7.0.76_1/temp
Using JRE_HOME: /usr/local/jdk1.7.0_80/
Using CLASSPATH: /home/tomcat/apache-tomcat-7.0.76_1/bin/bootstrap.jar:/home/tomcat/apache-tomcat-7.0.76_1/bin/tomcat-juli.jar
Using CATALINA_PID: /home/tomcat/apache-tomcat-7.0.76_1/bin/tomcat.pid
Tomcat started.

# /etc/init.d/tomcat2 start
Using CATALINA_BASE: /home/tomcat/apache-tomcat-7.0.76_2
Using CATALINA_HOME: /home/tomcat/apache-tomcat-7.0.76_2
Using CATALINA_TMPDIR: /home/tomcat/apache-tomcat-7.0.76_2/temp
Using JRE_HOME: /usr/local/jdk1.7.0_80/
Using CLASSPATH: /home/tomcat/apache-tomcat-7.0.76_2/bin/bootstrap.jar:/home/tomcat/apache-tomcat-7.0.76_2/bin/tomcat-juli.jar
Using CATALINA_PID: /home/tomcat/apache-tomcat-7.0.76_2/bin/tomcat.pid
Tomcat started.

# ps -ef | grep -i tomcat
tomcat 899 1 9 14:43 ? 00:00:10 /usr/local/jdk1.7.0_80//bin/java -Djava.util.logging.config.file=/home/tomcat/apache-tomcat-7.0.76_2/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.endorsed.dirs=/home/tomcat/apache-tomcat-7.0.76_2/endorsed -classpath /home/tomcat/apache-tomcat-7.0.76_2/bin/bootstrap.jar:/home/tomcat/apache-tomcat-7.0.76_2/bin/tomcat-juli.jar -Dcatalina.base=/home/tomcat/apache-tomcat-7.0.76_2 -Dcatalina.home=/home/tomcat/apache-tomcat-7.0.76_2 -Djava.io.tmpdir=/home/tomcat/apache-tomcat-7.0.76_2/temp org.apache.catalina.startup.Bootstrap start
tomcat 910 1 8 14:43 ? 00:00:08 /usr/local/jdk1.7.0_80//bin/java -Djava.util.logging.config.file=/home/tomcat/apache-tomcat-7.0.76_1/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.endorsed.dirs=/home/tomcat/apache-tomcat-7.0.76_1/endorsed -classpath /home/tomcat/apache-tomcat-7.0.76_1/bin/bootstrap.jar:/home/tomcat/apache-tomcat-7.0.76_1/bin/tomcat-juli.jar -Dcatalina.base=/home/tomcat/apache-tomcat-7.0.76_1 -Dcatalina.home=/home/tomcat/apache-tomcat-7.0.76_1 -Djava.io.tmpdir=/home/tomcat/apache-tomcat-7.0.76_1/temp org.apache.catalina.startup.Bootstrap start

10) 웹 어플리케이션 배포시...

* 배포하는 WEB Applicaion 의 web.xml 에 <distributable /> 추가 한다.

<%@ page session="true" %>
<%= session.getId() %>

jsp 페이지에 위의 내용을 넣고 세션 아이디를 확인해 본다.

My LiFe StOrY

2017년 3월 28일 화요일

[EtC] Apache, Tomcat Load Balancing & Session Clustering

2017년 3월 23일 목요일

[EtC] log4j log level