Load Balancing

Solrs supports load balancing of queries over multiple solr servers. There are 2 load balancers provided out of the box, a simple round robin LB and statistics based LB that selects the “fastest” server. If none of them is suitable for you, of course you can write your own load balancer by implementing io.ino.solrs.LoadBalancer.

Round Robin Load Balancer

The RoundRobinLB is a simple round robin load balancer. It load balances over a given list of solr server urls (if statically known), alternatively you can provide a SolrServers instance, which allows runtime resolution of solr server urls, e.g. by reading them from ZooKeeper (see Solr Cloud Support for more details).

To run solrs with a RoundRobinLB you have to pass it to the Builder

Java
RoundRobinLB lb = RoundRobinLB.create(asList(
  "http://localhost:8983/solr/collection1",
  "http://localhost:8984/solr/collection1"
));
JavaAsyncSolrClient solr = JavaAsyncSolrClient.builder(lb).build();
Full source at GitHub
Scala
val lb = RoundRobinLB(IndexedSeq(
  "http://localhost:8983/solr/collection1",
  "http://localhost:8984/solr/collection1"
))
val solr = AsyncSolrClient.Builder(lb).build
Full source at GitHub

Fastest Server Load Balancer

The FastestServerLB is a statistics based load balancer that classifies servers as “fast” and “slow” servers (based on their latest average response time) and selects one of the “fast” servers (round robin) when asked for one. This is useful e.g. when some solr server is currently performing major GC, or when for some nodes network latency is increased (temporary or permanent).

The latest average response time is determined in the following order (the first found measure is used):

  1. currently still running requests (if they’re lasting longer than previous, already completed requests)
  2. average response time of the current or the previous second
  3. average response time of the last ten seconds
  4. total average resonse time

The response time is measured using a configured test query (per collection). A dedicated test query is used, because user queries can have very different performance characteristics, so that most often it would even be hard for an application to classify them. With the dedicated test query you can control what is used to measure response time.

Servers are considered “fast” when the response time is <= the average response time of all servers. This is the default, you can also override this (by specifying a filterFastServers function).

Because nobody likes log spamming and burning CPU time while everybody else is sleeping, the test query is not executed with a fixed rate.
For “fast” servers test queries are run whenever a request comes in, with a lower bound of minDelay (default: 100 millis). With high traffic this leads to high resolution statistics so that e.g. sub-second GC pauses should be detected.
For “slow” servers (response time > average) tests are run with a fixed maxDelay (default: 10 seconds), this is also the case for “fast” servers when there are no users queries in the meantime.

To have initial stats, after the FastestServerLB was created it runs the test queries several times (default: 10). This can be overridden with initialTestRuns.

Hint

FastestServerLB also exports stats via JMX (under object name io.ino.solrs:type=FastestServerLB), in case you’re interested in this.

Here’s a code sample of the FastestServerLB:

Java
import io.ino.solrs.*;
import static java.util.Arrays.asList;
import scala.Tuple2;
import static java.util.concurrent.TimeUnit.*;

StaticSolrServers servers = StaticSolrServers.create(Arrays.asList(
  "http://localhost:8983/solr/collection1",
  "http://localhost:8984/solr/collection1"
));
Tuple2<String, SolrQuery> col1TestQuery = new Tuple2<>("collection1", new SolrQuery("*:*").setRows(0));
Function<SolrServer, Tuple2<String, SolrQuery>> collectionAndTestQuery = server -> col1TestQuery;
FastestServerLB<?> lb = FastestServerLB.builder(servers, collectionAndTestQuery)
        .withMinDelay(50, MILLISECONDS)
        .withMaxDelay(5, SECONDS)
        .withInitialTestRuns(50)
        .build();
JavaAsyncSolrClient solr = JavaAsyncSolrClient.builder(lb).build();
Full source at GitHub
Scala
import io.ino.solrs._
import io.ino.solrs.future.ScalaFutureFactory.Implicit
import scala.concurrent.duration._

val lb = {
  val servers = StaticSolrServers(IndexedSeq(
    "http://localhost:8983/solr/collection1",
    "http://localhost:8984/solr/collection1"
  ))
  val col1TestQuery = "collection1" -> new SolrQuery("*:*").setRows(0)
  def collectionAndTestQuery(server: SolrServer) = col1TestQuery
  new FastestServerLB(servers, collectionAndTestQuery, minDelay = 50 millis, maxDelay = 5 seconds, initialTestRuns = 50)
}
val solr = AsyncSolrClient.Builder(lb).build
Full source at GitHub
The source code for this page can be found here.