This is the fifth in a multi-part series where I explore the process of transforming an existing Graylog install into a resilient and scalable multi-site installation. Start here for Part 1.
At this point we have the dedicated ElasticSearch nodes up and running with the legacy node shut down. MongoDB has been split out and is in a Master/Slave replication mode running on the new nodes. Graylog2 Server is up and running on the new instances with collection inputs created. Graylog2 Web is running on all Graylog2 nodes. By now the only thing that should be left running on the legacy node is the primary Graylog2 Web instance and the log listener on the legacy Graylog2 server.
The final architectural piece to all this madness is to down the legacy Graylog2 instance and re-IP its replacement. Since all of the syslog devices ship their logs to the legacy server this will let us continue to receive logs while we rebalance all the end points.
Our primary goal here is to make sure all of the configs are up to date before we re-IP the server and then restart all the services.
Updating Graylog2 Server
/etc/graylog2.conf and confirm the settings:
west-gray01 is replacing the legacy server we'll make it the master Graylog2 instance
make sure it points at the two new Graylog2 servers.
The ElasticSearch nodes should be mostly configured correctly already. For the most part we just
want to make sure the legacy node is removed. On both
west-es01 make sure