понедельник, 6 ноября 2017 г.

clojure monorepo how-to

Monorepo is the way to manage several related projects from one point. In this note I'll show you how to organize monorepo using Amperity plugin lein-monolith .

0. Lets create root project and some subprojects
  rootproject
                    project.clj        ;;project file for root project
                    /apps
                            /app01     ;; project folder for app01 project
                    /libs
                            /lib01       ;; project folder for lib01 project which is needed for app01
                            /lib02       ;; some other lib


1. We should put section :plugins [[lein-monolith "1.0.1"]] to root project.clj and to project.clj of all subprojects.

(defproject rootproject/all "MONOLITH"
  :description "Mono repo"

  :plugins [[lein-monolith "1.0.1"]
                 [lein-cprint "1.3.0"]]

;; some repo which is used in all subprojects
  :repositories {"readytalk" {:url "https://dl.bintray.com/readytalk/maven/"}}

 ;; this is local deps that belongs only root project until section :dependencies is included in :inherit section. But for :inherit section we will use :managed-dependencies instead.
  :dependencies [[org.clojure/clojure "1.9.0-beta4"]]

;; this managed-deps will be delivered to all subprojects
  :managed-dependencies [[org.clojure/clojure "1.9.0-beta4"]]
 
  :monolith {
             ;; here we tell lein-monolith to merge :repositories and :managed-dependencies
             ;; to all subprojects.
             :inherit [:repositories :managed-dependencies]

            ;; here we tell that section profiles from root projects should be merged to all subprojects.
            ;; in this example we tell that in uberjar we should omit source files.
             :inherit-leaky [:profiles]

             ;; where is our subprojects located
             :project-dirs ["apps/*" "libs/*"]}

  :profiles {:uberjar {:omit-source true}}

  :env {:foo "bar"})

2. Every project.clj of all subprojects we prepare like this:

(defproject rootproject/app01 "2.0.0-SNAPSHOT"
  :description "FIXME: write description"

;; here we include lein-monolith for subproject
  :plugins [[lein-monolith "1.0.1"]
                 [lein-cprint "1.3.0"]]

 ;; here we tell that we need to inherit all sections from root project
  :monolith/inherit true
  :deployable true

  :dependencies [[rootproject/lib01 "0.1.0-SNAPSHOT"]]
  :main ^:skip-aot lib01.core
  :target-path "target/%s"
  :profiles {:uberjar {:aot :all}})

3. Now we can run commands from root project
lein monolith each do clean, uberjar 
This command will clean and make uberjars for all subprojects. Remember, that we should run
lein install
for lib01 to install to a local repo for app01 which use it. Also we can run command only for particular subproject
lein monolith each :start rootproject/app01 do uberjar 
 This command will run uberjar target only for particular project.

Interesting fact that we can point out clojure version only in root project - project.clj, in section :managed-dependencies and this version of clojure will be delivered to all subprojects. So we don't need to include clojure version in subprojects.

Also from root project we can run different tests for different subproject using selectors. First of all wee should put metadata to all deftest like here
https://github.com/technomancy/leiningen/blob/2.4.3/src/leiningen/test.clj#L164

In example above it is integrations test with ^:integration metadata. Then we should put section
:test-selectors
{:unit (complement :integration)
:integration :integration}

in root project.clj and put :test-selectors to :inherit section.  

воскресенье, 15 октября 2017 г.

clojure debugging 2

(defmacro safe-eval
  "This macro is used to execute any function inside try-catch block.
  returns value or nil in case of Exception and prints debug info. "
  [& forms]
  `(try
     ~@forms
     (catch Exception e#
       (println "--------------")
       (println "file :" *file* \newline)
       (println "s-exp:" (quote ~@forms))
       (println "src  :" ~(meta &form))
       (println "=>" (.getMessage e#)))))

воскресенье, 8 октября 2017 г.

clojure debugging

Add to project.clj
;;scope capture for debugging
 [vvvvalvalval/scope-capture "0.1.0"]

(require 'sc.api)

(defn abc [x]
  (let [y (* 2 x)
        z (Math/pow x 4)]
    (sc.api/spy {:abc [x y z]})))

(abc 2)
;;prints during execution: SPY [1 -2]  where 1 is id for defsc

(sc.api/defsc 1)

;;show me the state of local vars from let using given id 4
(sc.api/letsc 4 [x y z])
;; recreate the last operation result from let using given id 2
(sc.api/letsc 2 {:abc [x y z]})

;;show me the info map at the spy execution point using id 2
(sc.api/ep-info 2)

see video: https://vimeo.com/237220354

воскресенье, 1 октября 2017 г.

network async communication

Nikita made a good library [net.async/async "0.1.0"] for asynchronous network communication.
https://github.com/tonsky/net.async

Just add this to your project.clj

      [org.clojure/clojure "1.8.0"]
      ;; async network communications
      [net.async/async "0.1.0" :exclusions [[org.clojure/clojure]
                                            [org.clojure/tools.logging]
                                            [org.clojure/core.async]]]

      [org.clojure/core.async "0.3.443" :exclusions [org.clojure/tools.reader]]

      [org.clojure/tools.logging "0.4.0"]

Here is an output of example below:
-------------------------------
server:  4263
client:  ECHO/4263
-------------------------------
server:  16640
client:  ECHO/16640
-------------------------------
server:  75968
client:  ECHO/75968
-------------------------------
server:  11092
client:  ECHO/11092
-------------------------------
server:  61707
client:  ECHO/61707
-------------------------------
server:  50773
client:  ECHO/50773
-------------------------------
server:  16803
client:  ECHO/16803
-------------------------------

Here is a source code of echo server and client:

(require '[clojure.core.async :refer [<! >!  <!! >!! close! go]])
(use 'net.async.tcp)

(def event-l-client (event-loop))
(def event-l-server (event-loop))

(defn echo-server [evt-loop]
  (let [acceptor (accept evt-loop {:port 8899})]
    (loop []
      (when-let [server (<!! (:accept-chan acceptor))]
        (go
          (loop []
            (when-let [msg (<! (:read-chan server))]
              (when-not (keyword? msg)
                (>! (:write-chan server) (.getBytes (str "ECHO/" (String. msg))))
                (println "server: " (String. msg)))
              (recur))))
        (recur)))))


(defn echo-client [evt-loop]
  (let [client (connect evt-loop {:host "127.0.0.1" :port 8899})]
    (loop []
      (go (>! (:write-chan client) (.getBytes (str (rand-int 100000)))))
      (loop []
        (let [read (<!! (:read-chan client))]
          (when (bytes? read) (println "client: " (String. read)))
          (when (and (keyword? read)
                     (not= :connected read))
            (println "disconnected from server. trying to reconnect using timeout")
            (recur))))
      (println "-------------------------------")
      (Thread/sleep (rand-int 3000))
      (recur))))


(future (echo-server event-l-server))

(future (echo-client event-l-client))

(shutdown! event-l-server)
(shutdown! event-l-client)

среда, 23 августа 2017 г.

react learning: sorting table

(enable-console-print!)
(println "js app is started.")

(def excel-data (reagent/atom {:h ["book" "author" "language" "published" "sales"]
                               :b [["The Lord of the Rings" "J. R. R. Tolkien" "English" "1954–1955" "150 million"]
                                   ["Le Petit Prince (The Little Prince)" "Antoine de Saint-Exupéry" "French" "1943" "140 million"]
                                   ["Harry Potter and the Philosopher's Stone" "J. K. Rowling" "English" "1997" "107 million"]
                                   ["And Then There Were None" "Agatha Christie" "English" "1939" "100 million"]
                                   ["Dream of the Red Chamber" "Cao Xueqin" "Chinese" "1754–1791" "100 million"]
                                   ["The Hobbit" "J. R. R. Tolkien" "English" "1937" "100 million"]
                                   ["She: A History of Adventure" "H. Rider Haggard" "English" "1887" "100 million"]]
                               :sort-type {true > false <}
                               :sort-current true
                               :sort-column 0}))

(defn table-body
  [body]
  (for [row body]
    ^{:key (gensym)} [:tr (for [cell row]
                            ^{:key (gensym)}  [:td cell])]))

(defn change-sort-direction-if-needed
  [column-index]
  ;; when click on the same column then change sort direction
  (when (= column-index (:sort-column @excel-data))
    (swap! excel-data assoc-in [:sort-current] (not (:sort-current @excel-data)))))

(defn sort-table [column-index]
  (change-sort-direction-if-needed column-index)
  (let [sort-fn ((:sort-type @excel-data) (:sort-current @excel-data))
        sorted-data (sort-by #(nth % column-index) sort-fn (:b @excel-data))]
    (swap! excel-data assoc-in [:b] sorted-data)
    (swap! excel-data assoc-in [:sort-column] column-index)))

(defn table-header
  [header]
  [:tr
   (doall
    (for [header-item header]
      ^{:key (gensym)} [:th (if (= (.indexOf header header-item) (:sort-column @excel-data))
                              (str header-item (if (:sort-current @excel-data) \u2193 \u2191))
                              header-item)]))])

(defn excel-app
  "Excel prototype application"
  [excel-data]
  [:div
   [:table
    [:thead {:on-click #(sort-table (-> % .-target .-cellIndex))} (table-header (:h @excel-data))]
    [:tbody (table-body (:b @excel-data))]]])

(reagent/render-component [excel-app excel-data]
                          (.getElementById js/document "app"))



looks like this:



вторник, 22 августа 2017 г.

react learning: simple table

(enable-console-print!)
(println "js app is started.")

(def excel-data (reagent/atom {:h ["book" "author" "language" "published" "sales"]
                               :b [["The Lord of the Rings" "J. R. R. Tolkien" "English" "1954–1955" "150 million"]
                                   ["Le Petit Prince (The Little Prince)" "Antoine de Saint-Exupéry" "French" "1943" "140 million"]
                                   ["Harry Potter and the Philosopher's Stone" "J. K. Rowling" "English" "1997" "107 million"]
                                   ["And Then There Were None" "Agatha Christie" "English" "1939" "100 million"]
                                   ["Dream of the Red Chamber" "Cao Xueqin" "Chinese" "1754–1791" "100 million"]
                                   ["The Hobbit" "J. R. R. Tolkien" "English" "1937" "100 million"]
                                   ["She: A History of Adventure" "H. Rider Haggard" "English" "1887" "100 million"]]}))

(defn table-body
  [body]
  (for [row body]
    [:tr (for [cell row]
           [:td cell])]))

(defn table-header
  [header]
  [:tr (for [e header]
         [:th e])])

(defn excel-app
  "Excel prototype application"
  [excel-data]
  [:div
   [:table
    [:thead (table-header (:h @excel-data))]
    [:tbody (table-body (:b @excel-data))]]])

(reagent/render-component [excel-app excel-data]
                          (.getElementById js/document "app"))



It looks like this:



PS: Use ^{:key (gensym)} before :th, :tr, :td to avoid warning about unique key

воскресенье, 20 августа 2017 г.

react learning: shared state example

Using Reagent

(def str-area (reagent/atom "init"))
(def length (reagent/atom 0))

(defn my-text-area [area-value]
  [:textarea {:id :comments
              :defaultValue @area-value
              :on-change #(do
                            (reset! str-area (-> % .-target .-value))
                            (reset! length (count (-> % .-target .-value))))}])

(defn text-counter []
  [:div
   "Counter: " @length])

(defn text-value []
  [:div
   "Value: " @str-area])

(defn app []
  [:div
   [my-text-area str-area]
   [text-counter]
   [text-value]])

(reagent/render-component [app]
                          (.getElementById js/document "app"))

It looks like this:


Using RUM

(enable-console-print!)

(rum/defc count-label
  [value]
  [:div [:label (str "Count:" (count @value))]])

(rum/defcs app < (rum/local "hi!" ::value)
  [state]
  (let [v (::value state)]
    [:div
     [:div [:textarea {:default-value @v :on-change (fn [e] (reset! v (-> e .-target .-value)))}]]
     (count-label v)
     [:div [:label (str "Value:" @v)]]]))

(rum/mount (app) (.getElementById js/document "app"))




воскресенье, 6 августа 2017 г.

Enable Java Mission Control

Just put these flags as JVM parameters:

-Dcom.sun.management.jmxremote.rmi.port=7091 -Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -XX:+UnlockCommercialFeatures -XX:+FlightRecorder


среда, 12 июля 2017 г.

my if

;; what if "if" is unavailable?
;; what if we cannot use  code branch execution fn's? (case, cond, if, etc..)
;; let's create our own if


среда, 14 июня 2017 г.

Event Sourcing part 1

Spacemacs + smartscan

Very handy package - Smartscan for Clojure code editing, renaming local vars in fn, jumping to usages & more.
https://github.com/mickeynp/smart-scan


Spacemacs + Cider + 2 repls

When we use EMACS/Spacemacs and want to use two repls for CLJ and CLJS then
1. just add (add-to-list 'auto-mode-alist '("\.cljs$" . clojurescript-mode)) to init.el to enable clojurescript-mode for cljs files (otherwise it will be clojure mode with some problems in CLJS).
2. Select CLJ file and run cider-jack-in to run Clojure REPL
3. Select CLJS file (check that EMACS in Clojurescript mode) and run cider-jack-in-clojurescript or run cider-jack-in and then  (require'[ figwheel-sidecar.repl-api :as f]) and
(f/start-figwheel!) and (f/cljs-repl)

If I need to work with two different Clojure REPLs and eval S-forms I do following.

Run two REPLs by cider-jack-in command. 

To switch between REPL connections:
1. M-x and choose cider-toogle-request-dispatch to set CIDER variable to static mode
2. choose appropriate repl window and run command: cider-make-connection-default

Now, all S-exp will be evaluated in a default repl process.

воскресенье, 2 апреля 2017 г.

clojure.test tutorial

(ns example.basic-types-tests
  (:gen-class)
  (:require [clojure.test :refer [deftest testing is use-fixtures]]
            [example.basic-types :as basic]
            [clojure.spec :as s]
            [clojure.spec.gen :as gen]))


(defn once-fixture [f]
  (println "Prepare for for all tests...")
  (f)
  (println "Clean up after all tests..."))

(defn before-any-test-fixture [f]
  (println "Next test preparation...")
  (f)
  (println "Clean up after test..."))

(use-fixtures :once once-fixture)
(use-fixtures :each before-any-test-fixture)

(deftest basic-types-spec
  (testing "testing basic types spec"
    (is (= 1 1))))

(deftest basic-types-generator-test
  (testing "testing generators for basic types"
    (is (= 1000 (count (gen/sample (s/gen ::basic/string) 1000))))))

(deftest arithmetic-test
  (testing "this should throw exception"
    (is (thrown? ArithmeticException (/ 1 0)))))


;; This will make tests hook: only specified tests will be run.
;; Warining! if hook was installed then only specified tests will be run
;; to disable hook perform (ns-unmap *ns* 'test-ns-hook)
;(defn test-ns-hook []
;  (basic-types-generator-test))

;; To run particular test form some namespace use follwing command in repl:
;;(clojure.test/test-vars [#'example.basic-types-tests/arithmetic-test])
;; if everything ok it returns nil, otherwise returns report. This will run test with existing fixtures
;; or lein test :only my.namespace/my-test

суббота, 1 апреля 2017 г.

clojure.test with clojure.spec

First of all, add necessary dependency to project.clj
  [org.clojure/test.check "0.9.0"]

Then add following line to project.clj  :monkeypatch-clojure-test false due to bug https://github.com/technomancy/leiningen/issues/2173

Then, define fns and their spec

(require '[clojure.spec :as s])

(defn average [list-sum list-count]
  (/ list-sum list-count))

(s/fdef average
  :args (s/and (s/cat :list-sum float? :list-count integer?)
               #(not (zero? (:list-count %))))
  :ret number?)


Then in test folder make file with tests and put

(:require [clojure.test :as t]
            [clojure.string :as str]
            [example.core :refer :all]
            [clojure.spec :as spec]
            [clojure.pprint :as pprint]
            [clojure.spec.test :as stest])

(defmacro defspec-test
  ([name sym-or-syms] `(defspec-test ~name ~sym-or-syms nil))
  ([name sym-or-syms opts]
   (when t/*load-tests*
     `(def ~(vary-meta name assoc :test `(fn []
                                           (let [check-results# (clojure.spec.test/check ~sym-or-syms ~opts)
                                                 checks-passed?# (every? nil? (map :failure check-results#))]
                                             (if checks-passed?#
                                               (t/do-report {:type    :pass
                                                             :message (str "Generative tests pass for "
                                                                           (str/join ", " (map :sym check-results#)))})
                                               (doseq [failed-check# (filter :failure check-results#)
                                                       :let [r# (clojure.spec.test/abbrev-result failed-check#)
                                                             failure# (:failure r#)]]
                                                 (t/do-report
                                                   {:type     :fail
                                                    :message  (with-out-str (clojure.spec/explain-out failure#))
                                                    :expected (->> r# :spec rest (apply hash-map) :ret)
                                                    :actual   (if (instance? Throwable failure#)
                                                                failure#
                                                                (:clojure.spec.test/val failure#))})))
                                             checks-passed?#)))
        (fn [] (t/test-var (var ~name)))))))


(defspec-test average-test ['example.core/average] {:clojure.spec.test.check/opts {:num-tests 500000}})

Notice that testing fn is fully qualified and ' at the begining in vector.

That's it.


Update:

Also, we can write usual test. To test function example.core/average with input auto generation and output validation with spec we can do it followin way:

(t/deftest average-test2
  (t/testing "average test"  
    (let [r (stest/check `example.core/average {:clojure.spec.test.check/opts {:num-tests 1000}})]
        (t/is (= true (-> r first :clojure.spec.test.check/ret :result))))))

Update 2:

To test function example.core/average we need to require [clojure.test.check.clojure-test :as check.clj-test :refer [defspec]]

(defspec average-test3   10   (prop/for-all [r (gen/tuple gen/int gen/int)]
                                                    (number? (apply example.core/average r))))

Update 3:
To run reproducible test we need to require [clojure.test.check.clojure-test :as check.clj-test :refer [defspec]]

(defspec abcd {:seed 12345 :num-tests 1000}
         (prop/for-all [s (s/gen ::basic/blank-string)]
                       (s/valid? ::basic/blank-string s)))

Update 4:

In example.core namespace:

(defn my-div [x y]
  "divide two numbers"
  (/ x y))

(s/fdef my-div
  :args (s/and (s/cat :first number? :second number?)
               #(not (zero? (:second %))))
  :ret number?)

In tests namespace:

(deftest my-div-test
  (let [result (spec.test/check `example.core/my-div {:clojure.spec.test.check/opts {:num-tests 1000}})]
    (is (= true (-> result first :clojure.spec.test.check/ret :result)))
    (clojure.spec.test/instrument 'example.core/my-div)
    (is (thrown? clojure.lang.ExceptionInfo (example.core/my-div 1 0)))
    (clojure.spec.test/unstrument example.core/my-div)
    (is (thrown? ArithmeticException (example.core/my-div 1 0)))))




четверг, 16 марта 2017 г.

clojure queue FIFO

If you need simple FIFO queue you can use clojure.lang.PersistentQueue/EMPTY function with some wrappers on it.

;; ------------------------------------------------------------------
  (defn new-queue []
    (atom clojure.lang.PersistentQueue/EMPTY))

  (defn push-queue [queue value]
    (swap! queue conj value))

  (defn pop-queue [queue]
    (let [value (peek @queue)]
      (swap! queue pop)
      value))
;; ------------------------------------------------------------------

Now let's test it

  (def a (new-queue))
;; => nil

(push-queue a 1)
;; => #object...

(push-queue a 2)
;; => #object...

(push-queue a 3)
;; => #object...

(push-queue a "hello")
;; => #object...

Let's check a content of queue a:

(seq @a)
;; => (1 2 3 "hello")

Now time to take first item:

(pop-queue a)
;; => 1

(seq @a)
;; => (2 3 "hello")

понедельник, 13 марта 2017 г.

emacs interactive package install

(use-package company
:ensure t
:bind (("C-c /". company-complete))
:config
(global-company-mode))

четверг, 9 марта 2017 г.

воскресенье, 26 февраля 2017 г.

clojure reducers are fast

(def v1 (for [x (range 10000000)]
          (rand-int 10000000)))

(count v1)

(require '[clojure.core.reducers])
(defn x-in-coll? [x coll]
  (clojure.core.reducers/reduce (fn [res next-v] (if (= next-v x) (reduced true) false)) false coll))

(defn x-in-coll2? [x coll]
  (some #(= x %) coll))

(time (x-in-coll? 0 v1))
(time (x-in-coll2? 0 v1))

(time (x-in-coll? 0 v1))
"Elapsed time: 176.250871 msecs"
=> false
(time (x-in-coll2? 0 v1))
"Elapsed time: 559.220763 msecs"
=> nil

четверг, 23 февраля 2017 г.

netstat analog for mac os

lsof -iTCP -sTCP:LISTEN

lsof -iUDP

lsof -i4 -sTCP:LISTEN

lsof -i6 -sTCP:LISTEN


вторник, 21 февраля 2017 г.

clojure.spec for any namespace on the fly

(create-ns 'com.mycompany)
(alias 'a 'com.mycompany)
(s/def ::a/keyword keyword?)
(s/exercise ::a/keyword)

четверг, 9 февраля 2017 г.

java Date to LocalDate

(defn inst2local-date
  "convert java.util.Date to java.time.LocalDate"  [^Date date]
  (-> date .toInstant (.atZone (ZoneId/systemDefault)) .toLocalDate))

четверг, 2 февраля 2017 г.

Joda to LocalDate in Clojure

(def joda-local-date (t/local-date-time year month day))
(-> joda-local-date tc/to-long Instant/ofEpochMilli (.atZone (ZoneId/systemDefault)) .toLocalDate)

воскресенье, 29 января 2017 г.

clojurescript + reagent = track mouse example

Here is pretty simple example of mouse tracking app

(ns hello-world-cljs.core
  (:require [reagent.core :as r]))

(enable-console-print!)
(println "Hello App is started!")
;----------------------------------------------------------
(defonce app-state (r/atom {:mouse-trac? false :pointer {:x 0 :y 0}}))
;----------------------------------------------------------
(defn mouse-coords [e] {:x (.-pageX e) :y (.-pageY e)})
(defn mouse-handler [e] (swap! app-state assoc-in [:pointer] (mouse-coords e)))
(defn start-mouse-listen [handler] (js/document.addEventListener "mousemove" handler))
(defn stop-mouse-listen [handler] (js/document.removeEventListener "mousemove" handler))

;----------------------------------------------------------
(defn trac-mouse-button []
  [:div   [:button {:type "button" :class "btn btn-default" :on-click (fn [e] (swap! app-state update-in [:mouse-trac?] not))}
    "Mouse track?"]])

(defn trac-button-state-label [] [:div [:label (str (:mouse-trac? @app-state))]])

(defn mouse-coords-label [] [:div [:label (str "Pointer moved to: " (str (:pointer @app-state)))]])

(defn mouse-changer
  "this fn called every time when state of :mouse-trac? is changed"  []
  [:div   (if (:mouse-trac? @app-state)
     (start-mouse-listen mouse-handler)
     (stop-mouse-listen mouse-handler))])

(defn app-hello [] [:div [trac-mouse-button] [trac-button-state-label] [mouse-coords-label] [mouse-changer]])

(r/render-component [app-hello]
                    (.getElementById js/document "app"))


Here is result: