Apache Solr is the search platform that we install and maintain for most of our clients. We have found that no matter what the clients’ needs are, we can do it using Solr.
Historically, we have configured Solr in the master-slave configuration. This setup is quite durable and configurable. In this configuration, each Solr server is designated either as a master or a slave. The slave instances can take over the serving of search queries if a master becomes unavailable, meaning that there is no Single Point of Failure when it comes to serving queries and keeping your application up and running.
However, with the master-slave configuration, when a master server becomes unavailable, indexing becomes impossible until the master is restored. So, while the master-slave configuration is very durable in keeping your site up, indexing can get interrupted, which is not ideal. SolrCloud supports sharding in addition to clustering or replication of indexes. If you’re not familiar with sharding, it is a way of increasing scalability and durability by distributing the search index into smaller groups. There are several different strategies for splitting those indexes, but that is a topic beyond the scope of this blog post.
Other features that are available in the master-slave configuration are load balancing and distributed queries. While these features are possible to achieve, they require a lot of manual configuration. Moving to SolrCloud makes these features easier to configure and maintain, and more powerful.
In our more recent installations, Artemis has been setting up SolrCloud as our standard Solr installation. SolrCloud runs as a true cluster. Indexing occurs on all nodes, not just the master nodes. If any node fails, the cluster can continue to serve queries and perform indexes. Spinning up new nodes and adding them to the cluster is relatively easy as SolrCloud handles the workload and redistribution of the index shards. This is a huge advantage over the master-slave configuration.
In addition, queries to SolrCloud are routed to one of the nodes, which then aggregates results by querying other nodes and assembles the final result. While distributed queries can be set up in a master-slave configuration, it needs to be set up and tuned manually to achieve optimal performance. SolrCloud can optimize and balance the load between nodes transparently to the congress.gov web application.
If you have an existing Solr deployment using the master-slave configuration, you should consider moving to SolrCloud. To help you plan, here is a rough outline of the steps:
1. Deploy the new virtual hardware that will host the new SolrCloud nodes
2. Deploy new virtual machines to run Apache ZooKeeper nodes. We would need three (or five) nodes to avoid a Single Point of Failure with ZooKeeper. SolrCloud comes bundled with ZooKeeper – but the bundled version is not recommended for production environments
3. Migrate the schema.xml and solrconfig.xml files from the legacy system to SolrCloud. Both files will need to be updated to be valid for the new version of Solr. The best way to do this is to start with the valid files that come with the new version and migrate all of the configurations over manually, making sure each one works
4. Once it’s all up and running, do a full global reindex.
Happy Indexing and Querying!