Development


Project Version

mvn versions:set versions:update-child-modules -DparsedVersion.majorVersion=2 -DparsedVersion.minorVersion=1 -DparsedVersion.incrementalVersion=0 -DbuildId=SNAPSHOT
mvn clean install

In this example, we update the project version to 2.1.0-SNAPSHOT.

Upgrading parent version of project

When a project parent version has changed. To update the project parent version to latest

mvn validate -DallowSnapshots=true versions:update-parent versions:update-child-modules
mvn clean install

Upgrading versions inside properties to latest

To update the version of dependencies specified in properties section of the pom to latest. Please note: maven-version-plugin is used to restrict the properties to be updated. Check “versions-maven-plugin.includeProperties” property.

mvn validate -DallowSnapshots=true versions:update-properties versions:update-child-modules
mvn clean install

Bugzilla has an Inbound Email Interface but setting up is not documented, at least I couldn’t find it.

Here is how I did it

  • Make sure all of the optional E-mail/MIME perl modules are installed as listed by checksetup.pl
  • install procmail and MTA of your choice I used postfix
    apt-get install procmail
  • Create a user and added to apache group
    useradd -m -G www-data bug-mail
  • Create /home/bug-mail/.procmail as follows
    $cat /home/bug-mail/.procmail
    LOGFILE=/home/bug-mail/procmail.log
    :0:
    * ^To:.*bug-mail@yourdomain.com
    |/path/to/bugzilla/email_in.pl -vvv

    or specify some defaults in the config

    $cat /home/bug-mail/.procmail
    LOGFILE=/home/bug-mail/procmail.log
    :0:
    * ^To:.*bug-mail@yourdomain.com
    |/path/to/bugzilla/email_in.pl -vvv --default product=TestProduct --override version=2.5

How it works

I am not an expert but this is my understanding of how this works.

  • Procmail will invoke the .procmail file everytime an email is received for the user.
  • “:0:” means the first rule in the file and use a local lock file
  • “* ^To:.*bug-mail@yourdomain.com” this condition means match emails with the bug-mail@yourdomain.com in the To section
  • “|/path/to/bugzilla/email_in.pl” invoke the email_in.pl and pass the email content to it
  • email_in.pl will process the email content received

That’s it, happy bug creations!!!

Verify Perl Version

Make sure your perl version is >= 5.8.1 as shown below.

#perl -v
This is perl, v5.8.8 built for i386-linux-thread-multi

Most Linux distributions comes with perl. If you don’t have it on yours, download and install it from corresponding distribution website.

Install MySQL Database

Make sure your MySQL version is >= 4.1.2 as shown below.

# mysql -V
mysql Ver 14.12 Distrib 5.0.77, for redhat-linux-gnu (i686) using readline 5.1

If you don’t have mysql, install it as using yum groupinstall, or based on LAMP install article, or based on mysql rpm article.

Install Apache

If you already have apache installed, make sure you are able to access it by using http://{your-ip-address}.
If you don’t have apache, install is using yum based on LAMP install article, or install apache from source.

Download latest Bugzilla tar ball

Download the latest stable release from bugzilla download page. Extract the bugzilla*.tar.gz file to the apache document root directory as shown below.

# cd ~
# wget http://ftp.mozilla.org/pub/mozilla.org/webtools/bugzilla-3.6.tar.gz
# cd /var/www/html # tar xvfz /usr/save/bugzilla-3.4.6.tar.gz

Execute the bugzilla checksetup.pl

Bugzilla checksetup.pl program will verify whether all the required perl modules are installed. This will also display a list of all missing bugzilla modules that needs to be installed.
You can run the checksetup.pl program as many times as you like until you’ve verified all the required perl modules are installed.
Following is the output of 1st run of checksetup.pl, where is has listed all the missing optional and required modules.

# cd /var/www/html/bugzilla-3.4.6
# ./checksetup.pl --check-modules
COMMANDS TO INSTALL OPTIONAL MODULES:
 
             GD: /usr/bin/perl install-module.pl GD
          Chart: /usr/bin/perl install-module.pl Chart::Base
    Template-GD: /usr/bin/perl install-module.pl Template::Plugin::GD::Image
     GDTextUtil: /usr/bin/perl install-module.pl GD::Text
        GDGraph: /usr/bin/perl install-module.pl GD::Graph
       XML-Twig: /usr/bin/perl install-module.pl XML::Twig
     MIME-tools: /usr/bin/perl install-module.pl MIME::Parser
    libwww-perl: /usr/bin/perl install-module.pl LWP::UserAgent
    PatchReader: /usr/bin/perl install-module.pl PatchReader
     PerlMagick: /usr/bin/perl install-module.pl Image::Magick
      perl-ldap: /usr/bin/perl install-module.pl Net::LDAP
    Authen-SASL: /usr/bin/perl install-module.pl Authen::SASL
     RadiusPerl: /usr/bin/perl install-module.pl Authen::Radius
      SOAP-Lite: /usr/bin/perl install-module.pl SOAP::Lite
    HTML-Parser: /usr/bin/perl install-module.pl HTML::Parser
  HTML-Scrubber: /usr/bin/perl install-module.pl HTML::Scrubber
Email-MIME-Attachment-Stripper: /usr/bin/perl install-module.pl Email::MIME::Attachment::Stripper
    Email-Reply: /usr/bin/perl install-module.pl Email::Reply
    TheSchwartz: /usr/bin/perl install-module.pl TheSchwartz
 Daemon-Generic: /usr/bin/perl install-module.pl Daemon::Generic
       mod_perl: /usr/bin/perl install-module.pl mod_perl2
 
YOU MUST RUN ONE OF THE FOLLOWING COMMANDS (depending on which database you use):
 
PostgreSQL: /usr/bin/perl install-module.pl DBD::Pg
     MySQL: /usr/bin/perl install-module.pl DBD::mysql
    Oracle: /usr/bin/perl install-module.pl DBD::Oracle
 
COMMANDS TO INSTALL REQUIRED MODULES (You *must* run all these commands and then re-run checksetup.pl):
 
    /usr/bin/perl install-module.pl CGI
    /usr/bin/perl install-module.pl Digest::SHA
    /usr/bin/perl install-module.pl Date::Format
    /usr/bin/perl install-module.pl DateTime
    /usr/bin/perl install-module.pl DateTime::TimeZone
    /usr/bin/perl install-module.pl Template
    /usr/bin/perl install-module.pl Email::Send
    /usr/bin/perl install-module.pl Email::MIME
    /usr/bin/perl install-module.pl Email::MIME::Encodings
    /usr/bin/perl install-module.pl Email::MIME::Modifier
    /usr/bin/perl install-module.pl URI

To attempt an automatic install of every required and optional module with one command, do:
/usr/bin/perl install-module.pl --all

Execute bugzilla install-module.pl

As suggested by the output of the checksetup.pl, you can execute the install-module.pl to install all bugzilla required and optional perl modules.

# /usr/bin/perl install-module.pl --all

Please review the output of the above install-module.pl to make sure everything got install properly. There is a possibility that some of the modules failed to install (may be because some required OS packages were missing).
Execute the checksetup.pl to verify whether all the modules got installed properly.
Following is the output of 2nd run of the checksetup.pl:

# ./checksetup.pl --check-modules
COMMANDS TO INSTALL OPTIONAL MODULES:
 
             GD: /usr/bin/perl install-module.pl GD
          Chart: /usr/bin/perl install-module.pl Chart::Base
    Template-GD: /usr/bin/perl install-module.pl Template::Plugin::GD::Image
     GDTextUtil: /usr/bin/perl install-module.pl GD::Text
        GDGraph: /usr/bin/perl install-module.pl GD::Graph
       XML-Twig: /usr/bin/perl install-module.pl XML::Twig
     PerlMagick: /usr/bin/perl install-module.pl Image::Magick
      SOAP-Lite: /usr/bin/perl install-module.pl SOAP::Lite
       mod_perl: /usr/bin/perl install-module.pl mod_perl2
 
YOU MUST RUN ONE OF THE FOLLOWING COMMANDS (depending on which database
you use):
 
PostgreSQL: /usr/bin/perl install-module.pl DBD::Pg
     MySQL: /usr/bin/perl install-module.pl DBD::mysql
    Oracle: /usr/bin/perl install-module.pl DBD::Oracle

Install missing Perl Modules

As we see from the above checksetup.pl output, some of the optional modules and required module installed was not completed when we ran the install-module.pl.
So, we have to install the missing modules manually one-by-one to figure out the issues and fix it one-by-one.
Refer to the “Troubleshooting Section” at the end for list of all the issues that I faced while installing the perl modules required for bugzilla (along with the solution on how to fix those issues).

Final checksetup.pl –check-modules verification

Execute checksetup.pl –check-modules again as shown below as final verification to make sure all the modules got installed successfully.

# ./checksetup.pl --check-modules
* This is Bugzilla 3.4.6 on perl 5.8.8
* Running on Linux 2.6.18-164.el5PAE #1 SMP Thu Sep 3 04:10:44 EDT 2009
 
Checking perl modules...
Checking for              CGI.pm (v3.21)   ok: found v3.49
Checking for          Digest-SHA (any)     ok: found v5.48
Checking for            TimeDate (v2.21)   ok: found v2.24
Checking for            DateTime (v0.28)   ok: found v0.55
Checking for   DateTime-TimeZone (v0.71)   ok: found v1.17
Checking for                 DBI (v1.41)   ok: found v1.52
Checking for    Template-Toolkit (v2.22)   ok: found v2.22
Checking for          Email-Send (v2.00)   ok: found v2.198
Checking for          Email-MIME (v1.861)  ok: found v1.903
Checking for Email-MIME-Encodings (v1.313)  ok: found v1.313
Checking for Email-MIME-Modifier (v1.442)  ok: found v1.903
Checking for                 URI (any)     ok: found v1.54
 
Checking available perl DBD modules...
Checking for              DBD-Pg (v1.45)    not found
Checking for           DBD-mysql (v4.00)   ok: found v4.013
Checking for          DBD-Oracle (v1.19)    not found
 
The following Perl modules are optional:
Checking for                  GD (v1.20)   ok: found v2.44
Checking for               Chart (v1.0)    ok: found v2.4.1
Checking for         Template-GD (any)     ok: found v1.56
Checking for          GDTextUtil (any)     ok: found v0.86
Checking for             GDGraph (any)     ok: found v1.44
Checking for            XML-Twig (any)     ok: found v3.34
Checking for          MIME-tools (v5.406)  ok: found v5.427
Checking for         libwww-perl (any)     ok: found v5.834
Checking for         PatchReader (v0.9.4)  ok: found v0.9.5
Checking for          PerlMagick (any)     ok: found v6.2.8
Checking for           perl-ldap (any)     ok: found v0.4001
Checking for         Authen-SASL (any)     ok: found v2.1401
Checking for          RadiusPerl (any)     ok: found v0.17
Checking for           SOAP-Lite (v0.710.06) ok: found v0.711
Checking for         HTML-Parser (v3.40)   ok: found v3.65
Checking for       HTML-Scrubber (any)     ok: found v0.08
Checking for Email-MIME-Attachment-Stripper (any)     ok: found v1.316
Checking for         Email-Reply (any)     ok: found v1.202
Checking for         TheSchwartz (any)     ok: found v1.10
Checking for      Daemon-Generic (any)     ok: found v0.61
Checking for            mod_perl (v1.999022) ok: found v2.000004

Create localconfig file using checksetup.pl

Execute checksetup.pl without any argument, which will create a localconfig file in the current directory. The localconfig file contains the key configuration parameters used by the bugzilla (for example, mysql db username and password).

# ./checksetup.pl
Reading ./localconfig...
 
This version of Bugzilla contains some variables that you may want to
change and adapt to your local settings. Please edit the file
./localconfig and rerun checksetup.pl.
 
The following variables are new to ./localconfig since you last ran
checksetup.pl:  create_htaccess, webservergroup, db_driver, db_host, db_name, db_user, db_pass, db_port, db_sock, db_check, index_html, cvsbin, interdiffbin, diffpath, site_wide_secret

Modify the localconfig file.

The only thing you need to modify the localconfig file is MySQL database db password by changing the $db_pass variable as shown below.

# vi ./localconfig
$db_pass = 'Bugs4All';

Modify /etc/my.cnf to increase bugzilla attachment size

Set the max_allowed_packet to 4M in the /etc/my.cnf to increase bugzilla attachment size.

# cat /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
 
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links=0
 
# Allow packets up to 4MB
max_allowed_packet=4M
 
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

Restart the mysqld after this change.

# service mysqld restart

Create bugs mysql user

Add bugzilla user (bugs) to the mysql database as shown below.

# mysql -u root -p
 
mysql> GRANT SELECT, INSERT,
UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES,
CREATE TEMPORARY TABLES, DROP, REFERENCES ON bugs.*
TO bugs@localhost IDENTIFIED BY 'Bugs4All';
 
mysql> FLUSH PRIVILEGES;

Create the bugzilla database

Execute the checksetup.pl (without any arguments) again to create the mysql bugzilla database. Since the localconfig file already exist, the second time when you execute the checksetup.pl, it will create the mysql database based on the information from localconfig file.

# ./checksetup.pl
 
Creating database bugs...
 
Building Schema object from database...
Adding new table bz_schema ...
Initializing the new Schema storage...
Adding new table attach_data ...
Adding new table attachments ...
Adding new table bug_group_map ...
Adding new table bug_see_also ...
Adding new table bug_severity ...
Adding new table bug_status ...
 
Inserting values into the 'priority' table:
Inserting values into the 'bug_status' table:
Inserting values into the 'rep_platform' table:
 
Creating ./data directory...
Creating ./data/attachments directory...
Creating ./data/duplicates directory...
 
Adding foreign key: attachments.bug_id -> bugs.bug_id...
Adding foreign key: attachments.submitter_id -> profiles.userid...
Adding foreign key: bug_group_map.bug_id -> bugs.bug_id...

Create bugzilla administrator account.

At the end of the ./checksetup.pl execution, it will detect that you don’t have an adminsitrator account and request you to enter administration login information as shown below.

Looks like we don't have an administrator set up yet. Either this is
your first time using Bugzilla, or your administrator's privileges
might have accidentally been deleted.
 
Enter the e-mail address of the administrator: ramesh@thegeekstuff.com
Enter the real name of the administrator: Ramesh Natarajan
Enter a password for the administrator account: NotRealPwd
Please retype the password to verify: welcome
ramesh@thegeekstuff.com is now set up as an administrator.
Creating default classification 'Unclassified'...
Creating initial dummy product 'TestProduct'...
 
Now that you have installed Bugzilla, you should visit the 'Parameters'
page (linked in the footer of the Administrator account) to ensure it
is set up as you wish - this includes setting the 'urlbase' option to
the correct URL.

Configure Apache

Configure Port and DocumentRoot
Listen 80
DocumentRoot "var/www/html"
#Find the line
DirectoryIndex index.html index.html.var
#Change it to
DirectoryIndex index.html index.html.var index.cgi
#Add the following lines
ScriptAlias /cgi-bin/ “/var/www/html/bugzilla/”
#Add the following lines
Options Indexes FollowSymLinks ExecCGI
#Add the following lines
AllowOverride All
PerlSwitches -w -T
PerlConfigRequire /var/www/html/bugzilla/mod_perl.pl
<Directory /var/www/html/bugzilla>
    AddHandler cgi-script .cgi
    Options +Indexes +ExecCGI
    DirectoryIndex index.cgi
    AllowOverride Limit FileInfo Indexes
</Directory>

restart apache

Apache for mod_perl

Rename the bugzilla directory. (i.e remove the version number in it)

# cd /var/www/html
# mv bugzilla-3.4.6/ bugzilla

Add the following two lines to httpd.conf

# tail -2 /etc/httpd/conf/httpd.conf
PerlSwitches -I/var/www/html/bugzilla -I/var/www/html/bugzilla/lib -w -T
PerlConfigRequire /var/www/html/bugzilla/mod_perl.pl

Verify the Group in httpd.conf matches the webservergroup in localconfig

# cd /var/www/html/bugzilla/
 
# grep webservergroup localconfig
$webservergroup = 'apache';
 
# grep Group /etc/httpd/conf/httpd.conf
Group apache

Final checksetup.pl execution

Execute the checksetup.pl again.

# ./checksetup.pl
Reading ./localconfig...
 
Removing existing compiled templates...
Precompiling templates...done.
Fixing file permissions...
 
Now that you have installed Bugzilla, you should visit the 'Parameters'
page (linked in the footer of the Administrator account) to ensure it
is set up as you wish - this includes setting the 'urlbase' option to
the correct URL.

Login to bugzilla and complete one time setup.

Start the apache, go to http://{your-ip-address}/bugzilla and login using the administrator account you created above.
From the bugzilla UI, at the footer -> Administration -> Parameters -> ‘Required Settings’ section -> Fill-out following information:
maintainer: mainadmin@domain.com

urlbase: http://{your-ip-address}/
Note: Depending on your setup, go to -> User Authentication -> and you might want to change requiredlogin and emailregexp parameter.

Troubleshooting Bugzilla Install Issues

Issue1: DBD::mysql module failed

The DBD:mysql perl module failed with the “mysql.h: No such file or directory” error message as shown below.

# /usr/bin/perl install-module.pl DBD::mysql
 
dbdimp.h:22:49: error: mysql.h: No such file or directory
dbdimp.h:23:45: error: mysqld_error.h: No such file or directory
dbdimp.h:25:49: error: errmsg.h: No such file or directory
In file included from dbdimp.c:20:
dbdimp.h:144: error: expected specifier-qualifier-list before âMYSQLâ
dbdimp.h:236: error: expected specifier-qualifier-list before âMYSQL_RESâ

Solution1: install mysql-devel

Error message “mysql.h: No such file or directory” is because mysql-devel package was missing as shown below.

# /usr/bin/perl install-module.pl DBD::mysql
 
dbdimp.h:22:49: error: mysql.h: No such file or directory
dbdimp.h:23:45: error: mysqld_error.h: No such file or directory
dbdimp.h:25:49: error: errmsg.h: No such file or directory
In file included from dbdimp.c:20:
dbdimp.h:144: error: expected specifier-qualifier-list before âMYSQLâ
dbdimp.h:236: error: expected specifier-qualifier-list before âMYSQL_RESâ

Install the mysql-devel package as shown below.

# yum install mysql-devel
 
# rpm -qa | grep -i "mysql-devel"
mysql-devel-5.0.77-4.el5_4.2

DBD::mysql installation will go through without any issues now.

# /usr/bin/perl install-module.pl DBD::mysql

Issue2: GD failed with missing gdlib-config / libgd

Installing GD module failed with the following error message.

# /usr/bin/perl install-module.pl GD
 
**UNRECOVERABLE ERROR**
Could not find gdlib-config in the search path. Please install libgd 2.0.28 or higher.
If you want to try to compile anyway, please rerun this script with the option --ignore_missing_gd.
Running make test
  Make had some problems, maybe interrupted? Won't test
Running make install
  Make had some problems, maybe interrupted? Won't install

Solution2: Install gd-devel package

Install libgd (i.e gd-devel package) as shown below to fix the GD module issue.

# yum install gd-devel
 
# rpm -qa | grep gd
gd-2.0.33-9.4.el5_4.2
gd-devel-2.0.33-9.4.el5_4.2

GD got installed without any issues after insingalling gd-devel package.

# /usr/bin/perl install-module.pl GD

Issue3: Twig Failed with expat.h error

Twig module failed to install with the error message “expat.h: No such file or directory” as shown below.

# /usr/bin/perl install-module.pl XML::Twig
 
Expat.xs:12:19: error: expat.h: No such file or directory
Expat.xs:60: error: expected specifier-qualifier-list before XML_Parser

Solution3: Install expat and expat-devel for Twig

Install expat and expat-devel package as shown below.

# yum install expat
 
# yum install expat-devel

Now install Twig without any issues.

# /usr/bin/perl install-module.pl XML::Twig

Issue4: Image::Magick failed to install

Image::Magick installation failed with “magick/MagickCore.h: No such file or directory” error message as shown below.

# /usr/bin/perl install-module.pl Image::Magick
 
Note (probably harmless): No library found for -lMagickCore
Magick.xs:64:31: error: magick/MagickCore.h: No such file or directory
Magick.xs:171: error: expected specifier-qualifier-list before ‘MagickRealType’
Magick.xs:192: error: expected specifier-qualifier-list before ‘ImageInfo’
Magick.xs:214: error: ‘MagickNoiseOptions’ undeclared here (not in a function)
Magick.xs:214: warning: missing initializer

Solution4: Image::Magick failed to install

Make sure following ImageMagic related packages are present.

# /usr/bin/perl install-module.pl Image::Magick
 
Note (probably harmless): No library found for -lMagickCore
Magick.xs:64:31: error: magick/MagickCore.h: No such file or directory
Magick.xs:171: error: expected specifier-qualifier-list before ‘MagickRealType’
Magick.xs:192: error: expected specifier-qualifier-list before ‘ImageInfo’
Magick.xs:214: error: ‘MagickNoiseOptions’ undeclared here (not in a function)
Magick.xs:214: warning: missing initializer

In my case, ImageMagic-devel was missing. So, installed it as shown below. After that, Image::Magick perl module got installed successfully.

# yum install ImageMagick-devel
 
# /usr/bin/perl install-module.pl Image::Magick

Issue5: SOAP::Lite failed to install

SOAP::Lite module failed to install with “Cannot locate version.pm in @INC” message as shown below.

#/usr/bin/perl install-module.pl SOAP::Lite
 
Failed test 'use SOAP::Lite;' at t/SOAP/Data.t line 5.
Tried to use 'SOAP::Lite'.
Error:  Can't locate version.pm in @INC

Solution5: Install version.pm required for SOAP::Lite

Installed version.pm as shown below. After this, SOAP::Lite got installed without any issue.

# perl -MCPAN -e 'install version'
 
# /usr/bin/perl install-module.pl SOAP::Lite

Issue6 (and Solution6): mod_perl was missing

Don’t install mod_perl using /usr/bin/perl install-module.pl mod_perl2 . Insetad, use yum to install mod_perl as shown below.

# yum install mod_perl

Issue7: Apache start failed

Starting apache failed with “Cannot locate Template/Config.pm in @INC” error message.

# service httpd restart
Stopping httpd:                                            [  OK  ]
 
Starting httpd: Syntax error on line 994 of /etc/httpd/conf/httpd.conf:
Can't locate Template/Config.pm in @INC

Solution7: Install Template-Tool Kit as shown below

Install Template-Tool kit to fix the above apache error message

# cpan
cpan> i /Template-Toolkit/
Distribution    A/AB/ABEL/Eidolon-Driver-Template-Toolkit-0.01.tar.gz
Distribution    A/AB/ABW/Template-Toolkit-1.07.tar.gz
Distribution    A/AB/ABW/Template-Toolkit-2.22.tar.gz
Distribution    I/IN/INGY/Template-Toolkit-Simple-0.03.tar.gz
4 items found
 
cpan> install A/AB/ABW/Template-Toolkit-2.22.tar.gz

Issue8: Apache start failed again

Starting apache failed with “Cannot locate DateTime/Locale.pm in @INC” error message.

# service httpd restart
Stopping httpd:                                            [  OK  ]
 
Starting httpd: Syntax error on line 994 of /etc/httpd/conf/httpd.conf:
Can't locate DateTime/Locale.pm in @INC

Solution8: Install DateTime/Locale.pm as shown below

Install DateTime/Locale.pm to fix the above apache error message

# cpan
 
cpan> install DateTime:Locale

Also, in your apache error_log if you see Digest/SHA.pm issue, you should install it as shown below.

# tail -f /etc/httpd/logs/error_log
Can't locate Digest/SHA.pm in @INC (@INC contains:
 
# cpan
cpan> install Digest::SHA

Issue 8: Apache2::SizeLimit did not get installed

Solution 8:

sudo apt-get install libapache2-mod-perl2-dev libapache2-mod-perl2-doc
$ sudo /usr/bin/perl install-module.pl Apache2::SizeLimit

SQL Servers uses two different types of authentication

  1. SQL Server Authentication: is using users who are created on the SQL Server
  2. Windows Authentication: is using users from Active Directory

This tutorial utilises JTDS to connect SQL Server which allows Windows Authentication

  • Download JTDS from http://jtds.sourceforge.net and extract
  • Copy jtds-x.x.x.jar to <TOMCAT-INSTALL>/lib
  • Edit <TOMCAT-INSTALL>/conf/server.xml file and add following between Context tags for the web app that needs connection
    To connect using

    • SQL Server Authentication
      <Resource
      	name="jdbc/myDS"
      	auth="Container"
      	type="javax.sql.DataSource"
      	driverClassName="net.sourceforge.jtds.jdbc.Driver"
      	url="jdbc:jtds:sqlserver://server:1433/mydatabase"
      	username="sqluser"
      	password="XXXXXX"
      	maxActive="20"
      	maxIdle="10"
      	maxWait="-1"
      	validationQuery="SELECT 1"
      />
      
    • Windows Authentication
      <Resource
      	name="jdbc/myDS"
      	auth="Container"
      	type="javax.sql.DataSource"
      	driverClassName="net.sourceforge.jtds.jdbc.Driver"
      	url="jdbc:jtds:sqlserver://server:1433;DatabaseName=mydatabase;useCursors=false;domain=DOM;username=sqluser;password=XXXXXX;user=sqluser"
      	username="sqluser"
      	password="XXXXXX"
      	maxActive="20"
      	maxIdle="10"
      	maxWait="-1"
      	validationQuery="SELECT 1"
      />
      
  • Restart Tomcat

Initial Setup

First you have to create a new private key within an existing or new keystore using keytool:

keytool -genkeypair -keyalg rsa -keystore keystorename -storepass keystorepassword -alias my_new_key
  • to protect the new private key by its own keypassword within the keystore you have to add the keypass option
  • if you don’t use the keypass option the keystore password is used to protect the private key
  • accordingly in order to delete a key password (which is not possible with keytool btw), you have to copy the keystore password to the key password
  • if you want to use the key as a client certifcate use RSA as key algorithm, instead of default DSA, because many servers (e.g IBM MQ) only accept RSA client certs

The next task is to generate a CSR, even if you want to create a self signed certifcate:

keytool -certreq -alias my_new_key -keystore keystorename -storepass keystorepassword -file my_new.csr

Now it is time to either send the CSR to the CA of your choice, or to sign it by your own CA using openssl:

openssl x509 -req -in my_new.csr -CA my_ca.crt -CAkey my_ca.key -out my_new.crt -days 365 -CAcreateserial -CAserial my_ca.seq

In case your certificate is not selfsigned, you first have to import the certificate of the CA, and in case all intermediate CAs, that signed your CSR. After that, by importing the certficate the chain of trust will be established.

keytool -import -alias my_ca -file ca.crt -keystore keystorename -storepass keystorepassword

You have to enter either “yes” or use.

Finally the signed certificate has to be imported into the keystore using the same alias as the private key:

keytool -import -alias my_new_key -file my_new.crt -keystore keystorename -storepass keystorepassword

Examing the keystore

To see what’s inside any given keystore:

keytool -list -keystore keystorename

for example:

Keystore type: jks
Keystore provider: SUN

Your keystore contains 2 entries

my_key, Sep 26, 2007, keyEntry,
Certificate fingerprint (MD5): BA:22:E1:9E:9D:83:05:5A:99:42:5E:EF:62:77:DE:5A
my_ca, Sep 26, 2007, trustedCertEntry,
Certificate fingerprint (MD5): 4E:B5:B6:7A:02:F8:F8:6E:5E:79:FB:84:65:75:42:68

To get detailed information, like issuer for an alias use “-v”!

Change keystore passpharse

To change to keystore passphrase use the following keytool command:

keytool -storepasswd -keystore keystorename

If you use JDK 1.6 keytool you have to change the keypasswd for all private keys within the keystore as well!

OpenSSL and Keystores

A common task is to exchange keys and certificates between apache webserver, ssl loadbalancer or java application server such as tomcat or BEA Weblogic. This means to convert keys and certificates from PEM,DER or PKCS12 to or from java keystores. The standard keytool is able to import or export certificates, but there is no way to do so with private keys.

Export certifcate:

keytool -export -rfc -alias my_cert -file cert.crt -keystore keystorename -storepass keystorepassword

Import certificate:

keytool -import -alias my_cert -file cert.crt -keystore keystorename -storepass keystorepassword

Import private key:

In order to import an exisiting private key you first have to get and compile the ImportKey.java, see end of the page, tool. I added options to import keys and certs into an existing keystore as well as setting the keystore passphrase via the command line.

Usage: java ImportKey keyfile certfile [alias] [keystore] [keystore_passphrase]

The key has to be in DER format, which can be easily done with openssl:

openssl pkcs8 -topk8 -nocrypt -in key.pem -inform PEM -out key.der -outform DER

In case of a self signed certifcate use:

openssl x509 -in cert.pem -inform PEM -out cert.der -outform DER

If the certifcate is signed by a foreign CA or even signed by intermediate CA(s) use:

openssl crl2pkcs7 [-certfile ca_intermediate.pem] -certfile ca.pem -in cert.pem -inform PEM -out cert.der -outform DER -nocrl

This will create a PKCS#7 container using DER format including the correct certificate chain.

Then build a new keystore using both key and certificate:

java ImportKey key.der cert.der my_alias

Export private key:

This is based on information from Mark Foster’s wiki.

In order to export any private key from an existing keystore download and compile ExportPriv.java. After compiling it run:

java ExportPriv &gt; exported.key

The key will be exported into exported.key file in PKCS#8 PEM format. This can be converted into RSA format which is needed by apache with:

openssl pkcs8 -inform PEM -nocrypt -in exported.key -out exported_rsa.key

Various needful commands

Convert PEM to PKCS12

To create a pkcs12 container from a pem private key and cert use:

openssl pkcs12 -export -in cert.pem -inkey key.pem -out cred.p12

Export key and cert from PKCS12 to PEM

If you have a pkcs12 container and its passphrase. use the following command to extract the private key and client certificate only (-clcerts), without encrypting the exported private key again (-nodes):

openssl pkcs12 -in cred.p12 -out certkey.pem -nodes -clcerts

As you will probably notice, both key and certificate are combined into one file. If you need them seperatly you can either split the file using your favorite editor by simply save everything between (and including) each of the —–BEGIN—– and —–END—– lines to separate files or use the following two commands to export them seperatly:

openssl pkcs12 -in cred.p12 -out cert.pem -nodes -clcerts -nokeys
openssl pkcs12 -in cred.p12 -out key.pem -nodes -nocerts

Remove passphrase of private key

It will prompt for current passphrase:

openssl rsa -in oldkey.pem -out newkey.pem

Change passphrase of private key

It will prompt for old passphrase and twice for new one:

openssl rsa [-des3|-aes128] -in oldkey.pem -out newkey.pem

View details of a certificate signing request CSR

openssl req -noout -text -in server.csr

Links

Graphical Keytool Tool

import java.security.*;
import java.io.IOException;
import java.io.InputStream;
import java.io.FileInputStream;
import java.io.DataInputStream;
import java.io.ByteArrayInputStream;
import java.io.FileOutputStream;
import java.security.spec.*;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;
import java.util.Collection;
import java.util.Iterator;

/**
 * ImportKey.java
 *
 * &lt;p&gt;This class imports a key and a certificate into a keystore
 * (&lt;code&gt;$home/keystore.ImportKey&lt;/code&gt;). If the keystore is
 * already present, it is simply deleted. Both the key and the
 * certificate file must be in &lt;code&gt;DER&lt;/code&gt;-format. The key must be
 * encoded with &lt;code&gt;PKCS#8&lt;/code&gt;-format. The certificate must be
 * encoded in &lt;code&gt;X.509&lt;/code&gt;-format.&lt;/p&gt;
 *
 * &lt;p&gt;Key format:&lt;/p&gt;
 * &lt;p&gt;&lt;code&gt;openssl pkcs8 -topk8 -nocrypt -in YOUR.KEY -out YOUR.KEY.der
 * -outform der&lt;/code&gt;&lt;/p&gt;
 * &lt;p&gt;Format of the certificate:&lt;/p&gt;
 * &lt;p&gt;&lt;code&gt;openssl x509 -in YOUR.CERT -out YOUR.CERT.der -outform
 * der&lt;/code&gt;&lt;/p&gt;
 * &lt;p&gt;Import key and certificate:&lt;/p&gt;
 * &lt;p&gt;&lt;code&gt;java comu.ImportKey YOUR.KEY.der YOUR.CERT.der&lt;/code&gt;&lt;/p&gt;&lt;br /&gt;
 *
 * &lt;p&gt;&lt;em&gt;Caution:&lt;/em&gt; the old &lt;code&gt;keystore.ImportKey&lt;/code&gt;-file is
 * deleted and replaced with a keystore only containing &lt;code&gt;YOUR.KEY&lt;/code&gt;
 * and &lt;code&gt;YOUR.CERT&lt;/code&gt;. The keystore and the key has no password;
 * they can be set by the &lt;code&gt;keytool -keypasswd&lt;/code&gt;-command for setting
 * the key password, and the &lt;code&gt;keytool -storepasswd&lt;/code&gt;-command to set
 * the keystore password.
 * &lt;p&gt;The key and the certificate is stored under the alias
 * &lt;code&gt;importkey&lt;/code&gt;; to change this, use &lt;code&gt;keytool -keyclone&lt;/code&gt;.
 *
 * Created: Fri Apr 13 18:15:07 2001
 * Updated: Fri Apr 19 11:03:00 2002
 *
 * @author Joachim Karrer, Jens Carlberg
 * @version 1.1
 **/
public class ImportKey  {

    /**
     * &lt;p&gt;Creates an InputStream from a file, and fills it with the complete
     * file. Thus, available() on the returned InputStream will return the
     * full number of bytes the file contains&lt;/p&gt;
     * @param fname The filename
     * @return The filled InputStream
     * @exception IOException, if the Streams couldn't be created.
     **/
    private static InputStream fullStream ( String fname ) throws IOException {
        FileInputStream fis = new FileInputStream(fname);
        DataInputStream dis = new DataInputStream(fis);
        byte[] bytes = new byte[dis.available()];
        dis.readFully(bytes);
        ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
        return bais;
    }

    /**
     * &lt;p&gt;Takes two file names for a key and the certificate for the key,
     * and imports those into a keystore. Optionally it takes an alias
     * for the key.
     * &lt;p&gt;The first argument is the filename for the key. The key should be
     * in PKCS8-format.
     * &lt;p&gt;The second argument is the filename for the certificate for the key.
     * &lt;p&gt;If a third argument is given it is used as the alias. If missing,
     * the key is imported with the alias importkey
     * &lt;p&gt;The name of the keystore file can be controlled by setting
     * the keystore property (java -Dkeystore=mykeystore). If no name
     * is given, the file is named &lt;code&gt;keystore.ImportKey&lt;/code&gt;
     * and placed in your home directory.
     * @param args [0] Name of the key file, [1] Name of the certificate file
     * [2] Alias for the key.
     **/
    public static void main ( String args[]) {

        // change this if you want another password by default
        String keypass = &quot;changeit&quot;;

        // change this if you want another alias by default
        String defaultalias = &quot;acpcas&quot;;

        // change this if you want another keystorefile by default
        String keystorename = System.getProperty(&quot;keystore&quot;);

        if (keystorename == null)
            keystorename = &quot;keystore.ImportKey&quot;;

        // parsing command line input
        String keyfile = &quot;&quot;;
        String certfile = &quot;&quot;;
        if (args.length &lt; 2 || args.length&gt;5) {
            System.out.println(&quot;Usage: java comu.ImportKey keyfile certfile [alias] [keystore] [keystore_passphrase] &quot;);
            System.exit(0);
        } else {
            keyfile = args[0];
            certfile = args[1];
            if (args.length&gt;2)
                defaultalias = args[2];
            if (args.length&gt;3)
                keystorename = args[3];
            if (args.length&gt;4)
                keypass = args[4];
        }

        try {
            // initializing and clearing keystore
            KeyStore ks = KeyStore.getInstance(&quot;JKS&quot;, &quot;SUN&quot;);

            try {
		ks.load(new FileInputStream ( keystorename ), keypass.toCharArray());
                System.out.println(&quot;Using keystore-file : &quot;+keystorename);
	    } catch ( IOException e) {
                System.out.println(&quot;Creating keystore : &quot;+keystorename);
                ks.load( null , keypass.toCharArray());
                ks.store(new FileOutputStream ( keystorename  ), keypass.toCharArray());
            }

            // loading Key
            InputStream fl = fullStream (keyfile);
            byte[] key = new byte[fl.available()];
            KeyFactory kf = KeyFactory.getInstance(&quot;RSA&quot;);
            fl.read ( key, 0, fl.available() );
            fl.close();
            PKCS8EncodedKeySpec keysp = new PKCS8EncodedKeySpec ( key );
            PrivateKey ff = kf.generatePrivate (keysp);

            // loading CertificateChain
            CertificateFactory cf = CertificateFactory.getInstance(&quot;X.509&quot;);
            InputStream certstream = fullStream (certfile);

            Collection c = cf.generateCertificates(certstream) ;
            Certificate[] certs = new Certificate[c.toArray().length];

            if (c.size() == 1) {
                certstream = fullStream (certfile);
                System.out.println(&quot;One certificate, no chain.&quot;);
                Certificate cert = cf.generateCertificate(certstream) ;
                certs[0] = cert;
            } else {
                System.out.println(&quot;Certificate chain length: &quot;+c.size());
                certs = (Certificate[])c.toArray();
            }

            // storing keystore
            ks.setKeyEntry(defaultalias, ff,
                           keypass.toCharArray(),
                           certs );
            System.out.println (&quot;Key and certificate stored.&quot;);
            System.out.println (&quot;Alias:&quot;+defaultalias+&quot;  Password:&quot;+keypass);
            ks.store(new FileOutputStream ( keystorename ),
                     keypass.toCharArray());
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }

}// KeyStore

Un-comment the SSL Connector entry in the $CATALINA_BASE/conf/server.xml as follows;

Without Native Library

Tomcat uses JSSE SSL Connector by default, the connector entry would be as follows;
$CATALINA_BASE/conf/server.xml

<Connector port="8445" protocol="HTTP/1.1" SSLEnabled="true"
    socketFactory="org.apache.tomcat.net.SSLSocketFactory"
    maxThreads="150" scheme="https" secure="true"
    keyAlias="casalias" keystoreFile="/etc/tomcat6/keystore" keystorePass="changeit"
    clientAuth="false" sslProtocol="TLS" />

Get the certificate from Certificate Authority, then import the certificate on sso-server as follows:

/usr/lib/jvm/java # keytool -import -keystore /etc/tomcat6/keystore -alias casalias -file /home/user/sso.crt
Enter keystore password: ********
Owner: CN=server, DC=domain, DC=com
Issuer: CN=server, DC=domain, DC=com
Serial number: 69fb9d59dca5d781442ced0844ea3d2f
Valid from: Tue Sep 14 15:10:35 BST 2010 until: Mon Sep 14 15:18:38 BST 2015
Certificate fingerprints:
         MD5:  78:A4:5C:22:0E:11:8D:FB:90:0F:1E:4A:ED:88:3E:11
         SHA1: 8C:C9:4E:05:F0:54:13:B3:C7:10:DD:4E:0B:5C:77:31:48:5A:6D:13
Trust this certificate? [no]:  yes
Certificate was added to keystore
With Native Library

To improve the performance of the Apache Tomcat Server you could use the Apache Tomcat Native library.
To install the library just execute:

aptitude install libtcnative-1

and add the following line to the file $CATALINA_HOME/bin/setenv.sh (create the file if it does not exist):

cat /usr/share/tomcat6/skel/bin/setenv.sh
export LD_LIBRARY_PATH=/usr/lib:$LD_LIBRARY_PATH

To verify the library is running you will find the following message in $CATALINA_HOME/logs/catalina.out:

INFO: Loaded APR based Apache Tomcat Native library 1.1.19.

Without the Apache Tomcat Native library you will find a message like:

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production

When Apache Tomcat Library is in use Tomcat uses APR SSL Connector and would need a certificate and private key files

cat $CATALINA_BASE/conf/server.xml
<Connector port="8445" protocol="HTTP/1.1" maxHttpHeaderSize="8192"
    maxThreads="150"
    enableLookups="false" disableUploadTimeout="true"
    acceptCount="100" scheme="https" secure="true"
    SSLEnabled="true"
    SSLCertificateFile="/etc/tomcat6/ssl/server.crt"
    SSLCertificateKeyFile="/etc/tomcat6/ssl/server.key" />