Into
A quick and dirty overview on some of the capabilities of Clojure spec, test.check and transducers. With focus on generative data. Describing models with spec and then use test.check to generate entities from that model. Bulletproof transducers with the generated data.
Wont cover or explain details, instead show how it can be used.
Some examples where generated data can be used:
- Provide generated data through a Rest API endpoint and serve it a JSON. Good starting point before you have data in your app.
- Use in tests.
- In Clojures fdef's to bulletproof your functions.
Dependencies
Clojure Deps handles dependencies. The deps.edn looks like this:
{:deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/test.check {:mvn/version "0.10.0"}}}Require
Start by require the needed namespaces.
(require '[clojure.string :as str]
'[clojure.spec.alpha :as s]
'[clojure.spec.test.alpha :as stest]
'[clojure.test.check.generators :as gen])Specs
Specs with two models. ::person and ::person-with-age.
Each attribute have a generator that knows how to generate appropriate data with a likelihood.
(s/def ::non-blank-string (s/and string? (complement str/blank?)))
(s/def ::known-email-providers #{"gmail.com" "live.se" "protonmail.com"})
(s/def ::random-email-providers
(s/with-gen
(s/and string? #(re-matches #".+\..+" %))
#(gen/fmap (fn [[a b]] (str a "." b))
(gen/tuple
(s/gen ::non-blank-string)
(s/gen ::non-blank-string)))))
(s/def :person/name
(s/with-gen
::non-blank-string
#(gen/frequency [[3 (s/gen ::non-blank-string)]
[7 (gen/elements ["Ann" "Anna" "Hanna" "Johanna" "Erik"])]])))
(s/def :person/email
(s/with-gen
(s/and string? #(re-matches #".+@.+\..+" %))
#(gen/fmap
(fn [[a b]] (str a "@" b))
(gen/tuple
(s/gen ::non-blank-string)
(gen/frequency [[5 (s/gen ::known-email-providers)]
[5 (s/gen ::random-email-providers)]])))))
(s/def :person/born
(s/with-gen
(s/int-in 1800 2500)
#(gen/frequency [[5 (s/gen (s/int-in 1800 2020))]
[5 (gen/choose 1985 1987)]])))
(s/def ::person
(s/keys :req [:person/name
:person/email
:person/born]))
(s/def :person/age (s/int-in 1 150))
(s/def ::person-with-age
(s/merge
::person
(s/keys :req [:person/age])))Generate
Generate a single entity.
(gen/generate (s/gen ::person))Result.
#:person{:name "Johanna",
:email "5g29n6pytf1Ze4Wa47e5Ff2Wv@live.se",
:born 1986}Generate a collection with 100 entities.
(gen/sample (s/gen ::person) 100)Results are truncated.
(#:person{:name "Ann", :email "t@gmail.com", :born 1800}
#:person{:name "Ann", :email "1@1.V", :born 1985}
#:person{:name "E", :email "V@gmail.com", :born 1800}
#:person{:name "Erik", :email "c@live.se", :born 1987}
#:person{:name "Erik", :email "OSW@Xc.AO7D", :born 1987}
#:person{:name "Ann", :email "wKoGD@T34.6U2", :born 1985}
#:person{:name "C0q082", :email "q@e7RR7N.P5ED", :born 1800}
,,,)Validate
Lets validate data against a spec.
(s/valid? ::person
#:person{:name "Johanna",
:email "5g29n6pytf1Ze4Wa47e5Ff2Wv@live.se",
:born 1986})
;; => true
(s/valid? ::person
#:person{:name "Someone"})
;; => falseTransducers
Use a composition of transducers on data what we can generate. Generate 1000 persons, filter out persons that have a name that contains "ann" and are born in 1985 or 1986 and uses proton as a mail provider. Set their current age and take 3 of them.
(defn person-name-contains [s]
(filter (comp #(re-matches (re-pattern (str "(?i).*" s ".*")) %) :person/name)))
(defn person-born-in-years [years-set]
(filter (comp years-set :person/born)))
(defn person-email-provider [provider]
(filter (comp #(re-matches (re-pattern (str "(?i).*" provider ".*")) %) :person/email)))
(defn calc-person-age []
(map (fn [{:person/keys [born] :as p}] (assoc p :person/age (- 2020 born)))))
(def xf
(comp
(calc-person-age)
(person-name-contains "ann")
(person-born-in-years #{1985 1986})
(person-email-provider "protonmail")
(take 3)))
(into [] xf (gen/sample (s/gen ::person) 1000))This is the results.
[#:person{:name "Ann",
:email "Tz77Z1sFXL6kO1fnojYy8@protonmail.org",
:born 1985,
:age 35}
#:person{:name "Hanna",
:email "O@protonmail.org",
:born 1985,
:age 35}
#:person{:name "Anna",
:email "Jq29sl1l2yoLmL42W3whqO4i8YtVJ@protonmail.org",
:born 1986,
:age 34}]Test transducer composition
Lets use Clojure's fdef to test our composition of transducers.
anns-with-proton-mail will be evaluated 1000 times. Each time with 100 new
::person's and a random positive integer that arguments the pagination. On every
invocation the function should return a collection of ::person-with-age and the
collection should not be greater than our paginate value. If a return don't conform to
our rules an error will be thrown.
(defn anns-with-proton-mail [paginate xs]
(into
[]
(comp
(calc-person-age)
(person-name-contains "ann")
(person-born-in-years #{1985 1986})
(person-email-provider "protonmail")
(take paginate))
xs))
(s/fdef anns-with-proton-mail
:args (s/cat :paginate pos-int? :xs (s/coll-of ::person :count 100))
:ret (s/coll-of ::person-with-age)
:fn (fn [{{p :paginate xs :xs} :args res :res}]
(>= p (count res))))
(stest/check `anns-with-proton-mail)
Results from stest/check.
({:spec
#object[clojure.spec.alpha$fspec_impl$reify__2524 0x449e3c52
"clojure.spec.alpha$fspec_impl$reify__2524@449e3c52"],
:clojure.spec.test.check/ret
{:result true,
:pass? true,
:num-tests 1000,
:time-elapsed-ms 17372,
:seed 1590228136922},
:sym user/anns-with-proton-mail})