Load Balancing

Solrs supports load balancing of queries over multiple solr servers. There are 2 load balancers provided out of the box, a simple round robin LB and statistics based LB that selects the “fastest” server. If none of them is suitable for you, of course you can write your own load balancer by implementing io.ino.solrs.LoadBalancer.

Round Robin Load Balancer

The RoundRobinLB is a simple round robin load balancer. It load balances over a given list of solr server urls (if statically known), alternatively you can provide a SolrServers instance, which allows runtime resolution of solr server urls, e.g. by reading them from ZooKeeper (see Solr Cloud Support for more details).

To run solrs with a RoundRobinLB you have to pass it to the Builder

Java

copysourceRoundRobinLB lb = RoundRobinLB.create(asList(
  "http://localhost:8983/solr/collection1",
  "http://localhost:8984/solr/collection1"
));
JavaAsyncSolrClient solr = JavaAsyncSolrClient.builder(lb).build();

Scala

copysourceval lb = RoundRobinLB(IndexedSeq(
  "http://localhost:8983/solr/collection1",
  "http://localhost:8984/solr/collection1"
))
val solr = AsyncSolrClient.Builder(lb).build

By default, update requests are also load balanced. To send update requests to shard leaders, you should set isUpdatesToLeaders = true. Then still the isSendToLeaders property of the update request (default true) will be taken into account, i.e. if this would be false, then the update request would be load balanced round robin. In other words, only if both RoundRobinLB.isUpdatesToLeaders and IsUpdateRequest.isSendToLeaders are true, the update request will be sent to the shard leader.

Java

copysourceRoundRobinLB lb = RoundRobinLB.create(asList(
        "http://localhost:8983/solr/collection1",
        "http://localhost:8984/solr/collection1"
), /* isUpdatesToLeaders */ true);
JavaAsyncSolrClient solr = JavaAsyncSolrClient.builder(lb).build();

Scala

copysourceval lb = RoundRobinLB(IndexedSeq(
  "http://localhost:8983/solr/collection1",
  "http://localhost:8984/solr/collection1"
), isUpdatesToLeaders = true)
val solr = AsyncSolrClient.Builder(lb).build

Fastest Server Load Balancer

The FastestServerLB is a statistics based load balancer that classifies servers as “fast” and “slow” servers (based on their latest average response time) and selects one of the “fast” servers (round robin) when asked for one. This is useful e.g. when some solr server is currently performing major GC, or when for some nodes network latency is increased (temporary or permanent).

The latest average response time is determined in the following order (the first found measure is used):

1. currently still running requests (if they’re lasting longer than previous, already completed requests)
2. average response time of the current or the previous second
3. average response time of the last ten seconds
4. total average resonse time

The response time is measured using a configured test query (per collection). A dedicated test query is used, because user queries can have very different performance characteristics, so that most often it would even be hard for an application to classify them. With the dedicated test query you can control what is used to measure response time.

Servers are considered “fast” when the response time is <= the average response time of all servers. This is the default, you can also override this (by specifying a filterFastServers function).

Because nobody likes log spamming and burning CPU time while everybody else is sleeping, the test query is not executed with a fixed rate.
For “fast” servers test queries are run whenever a request comes in, with a lower bound of minDelay (default: 100 millis). With high traffic this leads to high resolution statistics so that e.g. sub-second GC pauses should be detected.
For “slow” servers (response time > average) tests are run with a fixed maxDelay (default: 10 seconds), this is also the case for “fast” servers when there are no users queries in the meantime.

To have initial stats, after the FastestServerLB was created it runs the test queries several times (default: 10). This can be overridden with initialTestRuns.

Similarly as for RoundRobinLB, with isUpdatesToLeaders you can configure the FastestServerLB to send updates to leaders if IsUpdateRequest.isSendToLeaders is set to true as well.

Here’s a code sample of the FastestServerLB:

Java

copysourceimport io.ino.solrs.*;
import static java.util.Arrays.asList;
import scala.Tuple2;
import static java.util.concurrent.TimeUnit.*;

StaticSolrServers servers = StaticSolrServers.create(Arrays.asList(
  "http://localhost:8983/solr/collection1",
  "http://localhost:8984/solr/collection1"
));
Tuple2<String, SolrQuery> col1TestQuery = new Tuple2<>("collection1", new SolrQuery("*:*").setRows(0));
Function<SolrServer, Tuple2<String, SolrQuery>> collectionAndTestQuery = server -> col1TestQuery;
FastestServerLB<?> lb = FastestServerLB.builder(servers, collectionAndTestQuery)
        .withMinDelay(50, MILLISECONDS)
        .withMaxDelay(5, SECONDS)
        .withInitialTestRuns(50)
        .withUpdatesToLeaders(true)
        .build();
JavaAsyncSolrClient solr = JavaAsyncSolrClient.builder(lb).build();

Scala

copysourceimport io.ino.solrs._
import io.ino.solrs.future.ScalaFutureFactory.Implicit
import scala.concurrent.duration._

val lb = {
  val servers = StaticSolrServers(IndexedSeq(
    "http://localhost:8983/solr/collection1",
    "http://localhost:8984/solr/collection1"
  ))
  val col1TestQuery = "collection1" -> new SolrQuery("*:*").setRows(0)
  def collectionAndTestQuery(server: SolrServer) = col1TestQuery
  new FastestServerLB(
    servers,
    collectionAndTestQuery,
    minDelay = 50 millis,
    maxDelay = 5 seconds,
    initialTestRuns = 50,
    isUpdatesToLeaders = true)
}
val solr = AsyncSolrClient.Builder(lb).build