Upgrading to Graylog 1.0 GA

A few things to keep in mind. First a name change. During the 0.X versions the software was called Graylog2. Starting with the release of 1.0 the name was changed go Graylog. I'm pretty happy about the change. Graylog2 was a bit of a mouthful and including version numbers in package names has always annoyed me anyway. However, changing the version number makes the upgrade a bit harder.

I would plan this upgrade for a time when you can expect to not receive logs. During part of the upgrade you will have two instances of Graylog installed which may conflict with each other.

Package Repository

If you followed my previous posts you know I deal with Ubuntu, but modifying the steps to something more RedHaty should be relatively easy. First thing's first, we have to change where we download the packages. Edit /etc/apt/sources.list.d/graylog2.list to look more like this

#file generated by puppet
# This is the TORCH hosted official repositories for Graylog2
deb https://packages.graylog2.org/repo/debian/ trusty 1.0

With this version they changed to HTTPS for package delivery so make sure you also have the apt-transport-https package installed.

Install the New Packages

Since the name of the software changed the packages also changed. This one's easy. Just do a

aptitude install graylog-web graylog-server

At this point we have both Graylog2 0.{whatever} and Graylog 1.0 installed.

Update Graylog Server Config

The configuration file path is very similar between the two, which makes it easy. Historically graylog2-server installed its config files in one of two locations. The primary config file is /etc/graylog2.conf and all other files are in /etc/graylog2/server/. Starting with Graylog 1.0 all server config files are stored in /etc/graylog/server/. The quick and dirty approach is to just

cp /etc/graylog2.conf /etc/graylog/server/server.conf

I would actually recommend checking a diff of the files first and make sure you don't need to update anything manually.

diff -y /etc/graylog2.conf /etc/graylog/server/server.conf
# Look for changes that need to be made
cp /etc/graylog2.conf /etc/graylog/server/server.conf
# Clean up new server.conf with any necessary modifications

There were a few references to graylog2 in file paths that needed to be corrected, but there was only one setting that desperately needed changed. With Graylog 1.0 there's a new option called a message journal that is used. If you don't specify a location in the config file it will default to /data/journal/ which may result in startup errors. I plopped it in the same corresponding location as the old message cache.

# Enable the disk based message journal.
message_journal_enabled = true

# The directory which will be used to store the message journal
# must not contain any other files than the ones created by Graylog
message_journal_dir = /var/lib/graylog-server/journal

Also make sure to copy over the node-id. Otherwise your Graylog 1.0 instance will show up as a new server node which could cause some problems down the road.

cp /etc/graylog2/server/node-id /etc/graylog/server/node-id

Update Graylog Web Config

The default web server configuration file name has changed from graylog2-web-interface.conf to web.conf which better matches Graylog Server. We're going to follow much the same process as above. First validate the web server configs and copy them over.

diff -y /etc/graylog2/web/graylog2-web-interface.conf /etc/graylog/web/web.conf
# Look for changes that need to be made
cp /etc/graylog2/web/graylog2-web-interface.conf /etc/graylog/web/web.conf
# Clean up new web.conf with any necessary modifications

On my system no other files needed modification, but you might want to check them out as well.

Recycle Services

Along with everything else the service names also changed. Cycle all the services to get the new instance up and running.

initctl stop graylog2-server
initctl stop graylog2-web
initctl start graylog-server
initctl start graylog-web

Cleanup

Since the service account user changed we'll go ahead and change ownership to the new user.

find / -user graylog2 -print0 | xargs -0 chown graylog:graylog
find / -user graylog2-web -print0 | xargs -0 chown graylog-web:graylog-web

Next kill all the old packages.

aptitude remove graylog2-server graylog2-web

Now delete all the old config files. Make sure you have backups, just in case.

rm -rf /etc/graylog2 /etc/graylog2.conf /etc/init/graylog2-server.conf /etc/init/graylog2-web.conf /etc/default/graylog2-server /etc/default/graylog2-web

Now we can kill all the old caches.

rm -rf /var/lib/graylog2-server/ /var/lib/graylog2-web/

Once your local log retention schedule has expired to ahead and wipe the application logs as well.

rm -rf /var/log/graylog2-web /var/log/graylog2-server

Now kill the old service accounts. They're completely unecessary at this point and can be removed.

userdel graylog2
userdel graylog2-web

Finalize

At this point we should be totally good to go with our new environment. Log in to the web interface and browse around. Make sure the inputs are listening where they should be. Verify that data is actually accessible. If not, well, I wish you the best of luck in your exciting new troubleshooting endeavors!