ElasticSearch supports clustering ; that is, you can have a series of distinct ElasticSearch instances work in a coordinated manner without much administrative intervention at all. Clustering ElasticSearch instances (or nodes) provides data redundancy as well as data availability.
Best of all, clustering in ElasticSearch, by default, doesn’t require any configuration – nodes discover each other. You can set up a cluster in about 60 seconds. Let me show you how!
First, download and unzip or untar the latest version of ElasticSearch . Next, copy the resultant ElasticSearch install directory (for example, mine is dubbed elasticsearch-0.90.3) into 3 different directories; for example, I’ve called my directories node-1, node-2, and node-3.
Next, open up three terminal windows and in each, change directories to a sequential node. Start the instance in the node-1 directory like so:
Starting up ElasticSearch <span class='line-number'>1</span>
<code class='bash'><span class='line'>./bin/elasticsearch -f
</span></code>
The -f forces the process to run in the foreground.
By default, ElasticSearch nodes will name themselves if you don’t provide a name (via the elasticsearch.yml configuration file). Thus, after you start the first instance, you should see something along the lines of:
A master node is created! <span class='line-number'>1</span>
<code class='bash'><span class='line'><span class="o">[</span>cluster.service<span class="o">]</span> <span class="o">[</span>Dionysus<span class="o">]</span> new_master <span class="o">[</span>Dionysus<span class="o">][</span>r7gbosdKSWGfTCgRPrS6vw<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9300<span class="o">]]</span>, reason: zen-disco-join <span class="o">(</span>elected_as_master<span class="o">)</span>
</span></code>
In the above log output, “Dionysus” is the automatic name chosen by ElasticSearch. Note the part about “new_master” for the cluster.service.
Next, go into the next terminal window, say node-2, and start that instance the same way (via the -f flag). You should see the 2nd instance (in my case, named Caiera) discover the master:
Node #2 discovers the master node <span class='line-number'>1</span>
<code class='bash'><span class='line'><span class="o">[</span>cluster.service<span class="o">]</span> <span class="o">[</span>Caiera<span class="o">]</span> detected_master <span class="o">[</span>Dionysus<span class="o">][</span>r7gbosdKSWGfTCgRPrS6vw<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9300<span class="o">]]</span>, added <span class="o">{[</span>Dionysus<span class="o">][</span>r7gbosdKSWGfTCgRPrS6vw<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9300<span class="o">]]</span>,<span class="o">}</span>, reason: zen-disco-receive<span class="o">(</span>from master <span class="o">[[</span>Dionysus<span class="o">][</span>r7gbosdKSWGfTCgRPrS6vw<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9300<span class="o">]]])</span>
</span></code>
See the detected_master statement above? And you should see the master, “Dionysus”, add the 2nd instance, “Caiera”, too (via the first terminal window):
The master node adds node #2 <span class='line-number'>1</span>
<code class='bash'><span class='line'><span class="o">[</span>cluster.service<span class="o">]</span> <span class="o">[</span>Dionysus<span class="o">]</span> added <span class="o">{[</span>Caiera<span class="o">][</span>Eh3DHlcRQhGxatGnUG8smA<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9301<span class="o">]]</span>,<span class="o">}</span>, reason: zen-disco-receive<span class="o">(</span>join from node<span class="o">[[</span>Caiera<span class="o">][</span>Eh3DHlcRQhGxatGnUG8smA<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9301<span class="o">]]])</span>
</span></code>
Lastly, start the 3rd instance in the 3rd window.
In this case, my 3rd node is dubbed “Phantom Rider” and it’ll discover the master:
Node #3 discovers the master <span class='line-number'>1</span>
<code class='bash'><span class='line'><span class="o">[</span>cluster.service<span class="o">]</span> <span class="o">[</span>Phantom Rider<span class="o">]</span> detected_master <span class="o">[</span>Dionysus<span class="o">][</span>r7gbosdKSWGfTCgRPrS6vw<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9300<span class="o">]]</span>, added <span class="o">{[</span>Dionysus<span class="o">][</span>r7gbosdKSWGfTCgRPrS6vw<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9300<span class="o">]]</span>,<span class="o">[</span>Caiera<span class="o">][</span>Eh3DHlcRQhGxatGnUG8smA<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9301<span class="o">]]</span>,<span class="o">}</span>, reason: zen-disco-receive<span class="o">(</span>from master <span class="o">[[</span>Dionysus<span class="o">][</span>r7gbosdKSWGfTCgRPrS6vw<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9300<span class="o">]]])</span>
</span></code>
And the master, will in turn, add “Phantom Rider” into the cluster:
The master node adds node #3 <span class='line-number'>1</span>
<code class='bash'><span class='line'><span class="o">[</span>cluster.service<span class="o">]</span> <span class="o">[</span>Dionysus<span class="o">]</span> added <span class="o">{[</span>Phantom Rider<span class="o">][</span>Sw1bDSbFQqeTq4M45qbNpg<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9302<span class="o">]]</span>,<span class="o">}</span>, reason: zen-disco-receive<span class="o">(</span>join from node<span class="o">[[</span>Phantom Rider<span class="o">][</span>Sw1bDSbFQqeTq4M45qbNpg<span class="o">][</span>inet<span class="o">[</span>/192.168.1.12:9302<span class="o">]]])</span>
</span></code>
Now that you have 3 ElasticSearch instances running, you can run a few RESTful commands to verify your cluster is operational.
First, in a new terminal window, run this command:
cURL to ascertain cluster nodes You should see a nicely formatting JSON response. It’ll look something like:
ElasticSearch nodes <span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<span class='line-number'>20</span>
<span class='line-number'>21</span>
<span class='line-number'>22</span>
<span class='line-number'>23</span>
<code class='json'><span class='line'><span class="p">{</span>
</span><span class='line'> <span class="nt">"cluster_name"</span> <span class="p">:</span> <span class="s2">"elasticsearch"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"nodes"</span> <span class="p">:</span> <span class="p">{</span> <span class="nt">"Eh3DHlcRQhGxatGnUG8smA"</span> <span class="p">:</span> <span class="p">{</span> <span class="nt">"hostname"</span> <span class="p">:</span> <span class="s2">"new-host-5.home"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"http_address"</span> <span class="p">:</span> <span class="s2">"inet[/192.168.1.12:9201]"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"name"</span> <span class="p">:</span> <span class="s2">"Caiera"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"transport_address"</span> <span class="p">:</span> <span class="s2">"inet[/192.168.1.12:9301]"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"version"</span> <span class="p">:</span> <span class="s2">"0.90.3"</span>
</span><span class='line'> <span class="p">},</span>
</span><span class='line'> <span class="nt">"Sw1bDSbFQqeTq4M45qbNpg"</span> <span class="p">:</span> <span class="p">{</span> <span class="nt">"hostname"</span> <span class="p">:</span> <span class="s2">"new-host-5.home"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"http_address"</span> <span class="p">:</span> <span class="s2">"inet[/192.168.1.12:9202]"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"name"</span> <span class="p">:</span> <span class="s2">"Phantom Rider"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"transport_address"</span> <span class="p">:</span> <span class="s2">"inet[/192.168.1.12:9302]"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"version"</span> <span class="p">:</span> <span class="s2">"0.90.3"</span>
</span><span class='line'> <span class="p">},</span>
</span><span class='line'> <span class="nt">"r7gbosdKSWGfTCgRPrS6vw"</span> <span class="p">:</span> <span class="p">{</span> <span class="nt">"hostname"</span> <span class="p">:</span> <span class="s2">"new-host-5.home"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"http_address"</span> <span class="p">:</span> <span class="s2">"inet[/192.168.1.12:9200]"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"name"</span> <span class="p">:</span> <span class="s2">"Dionysus"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"transport_address"</span> <span class="p">:</span> <span class="s2">"inet[/192.168.1.12:9300]"</span><span class="p">,</span>
</span><span class='line'> <span class="nt">"version"</span> <span class="p">:</span> <span class="s2">"0.90.3"</span>
</span><span class='line'> <span class="p">}</span>
</span><span class='line'> <span class="p">},</span>
</span><span class='line'> <span class="nt">"ok"</span> <span class="p">:</span> <span class="kc">true</span>
</span><span class='line'><span class="p">}</span>
</span></code>
