clj-hazelcast

Clojure library for Hazelcast, an open source clustering and highly scalable data distribution platform for Java.

TravisCI Status

Usage

clj-hazelcast comes pre-configured out of the box and allows adding members as an option. For example, The init function takes a map like {:peers [hostname1 hostname2 ...]}, and starts a peer node on them as the member of the cluster.

This library supports synchronization of clj-kryo (another open source library from Runa labs) serializable data structures across network. It also supports event-driven communication among them. A typical use case would to sync key/value pairs among the peers. For example:

Suppose you have following data in all nodes:

(def test-map (atom nil))

You can put new values to this map on host 1:

(hazelcast/put! @test-map :baz "foobar")

Then you can retrieve "foobar" from host 2:

(:baz (hazelcast/get-map "your.name.space.test-map"))

Furthermore, you can add a listener to that data to perform all sorts of callbacks. For example:

(let [events (atom [])
      listener-fn (fn [& event]
                    (swap! events conj event))
      listener (hazelcast/add-entry-listener! @test-map listener-fn)
      result (do
               (hazelcast/put! @test-map :baz "foobar")
               (hazelcast/put! @test-map :foo "bizbang")
               (Thread/sleep 5)
               (count @events))]
  (hazelcast/remove-entry-listener! @test-map listener)
  result)

Please refer to the included test for more detail.

Using the MapReduce Framework

The namespace clj-hazelcast.mr contains an abstraction to make it easier when dealing with map-reduce jobs.

Mapper

Runs the function f over the content.
f is a function of two arguments: key and value.
f must return a sequence of pairs like

[[key1 value1] [key2 value2] ...]

Sample Mapper

(defmapper sample-mapper [k v] [[k (+ 1 v)]

This is a simple mapper that gets each key-value pair and returns a pair of k and value + 1.

Reducer

Runs the reducer function rf over the content.

rf takes three arguments: key, value and an accumulator.

rf should return the new accumulator.

Sample Reducer

(defreducer sample-reducer [k v acc] (if (nil? acc) 1 (inc acc)))

This is a simple counting reducer.

Collator

Runs the collactor function f over the content.

f takes one argument, a sequence of Entry objects, where Entry instances are generated by the reducer's output:

[Entry1 Entry2 ...]

Sample Collator

(defcollator sample-collator [seq] (reduce + (vals seq)))

This is a simple collator that counts aggregates of all values from a map.

Combiner

Combiners are mostly the same as the reducers

Wordcount Example

Check 'clj-hazelcast.test.mr-test

Distributed Query

You can use the defpredicate macro to use clojure functions as predicates. A predicate function is a function with 2 parameters, a key and a value, returning a boolean result.

(require '[clj-hazelcast.query :as q])
(q/defpredicate stark? [k v] (.contains v "Stark"))
(q/values @query-map stark?)

License

• The use and distribution terms for this software are covered by the
• Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)
• which can be found in the file epl-v10.html at the root of this distribution.
• By using this software in any fashion, you are agreeing to be bound by
• the terms of this license.
• You must not remove this notice, or any other, from this software.