Friday, August 28, 2009

clojure parallel map

Map is a function that takes two parameters. A list and another function that will be executed for every element in the list:

(defn multiply-by-2[x]
  (* x 2))

(map multiply-by-2 [0 1 2 3 4 5 6 7 8 9])
; => (0 2 4 6 8 10 12 14 16 18)

The map funktion is a wonderful thing, because every computation for a list element can be parallelized.

Even though Clojure has the function pmap that does exactly this parallelization, I built the same functionality on my own to understand multithreading with Clojure and java.util.concurrent.

Here it is:
(import '(java.util.concurrent Executors))
(defn my-pmap [map-fn, l] 
  (let [len (count l)
   pool  (Executors/newFixedThreadPool len)
   tasks (map
           (fn [i](fn [] (map-fn (nth l i))))
           (range len))]

     (let [result (map 
                    (fn [future](.get future)) 
                    (.invokeAll pool tasks))]
       (.shutdown pool)
       result)))

(my-pmap multiply-by-2 [0 1 2 3 4 5 6 7 8 9])
; => (0 2 4 6 8 10 12 14 16 18)

; it yields the same result as
(pmap multiply-by-2 [0 1 2 3 4 5 6 7 8 9])