class: center, middle, main # Adventures in Homoiconicity ### An exploration by Franka Schmidt
### PolyConf 2017 in Paris --- layout: true class: normal, middle, center --- # Hi, I'm Franka :wave: ## * Platform engineer at Mapbox! ## * :cycle:, :code:, :live: in Berlin ## * ClojureBridge Organizer ### @franschm ---  # ❤️️ Support your local ClojureBridge ❤️️ ## or find a ScalaBridge, RailsGirls, LesbiansWhoTech, ... --- # #dzy --- # Code is Data. # Data is Code! --- # Homoiconicity # is # ⚡⚡⚡ Clojure's superpower!! ⚡⚡⚡ ---
## _Clojure is a *homoiconic* language,
which is a fancy term describing the fact that Clojure programs are represented by Clojure data structures._
.right[From [Clojure.org](https://clojure.org/reference/reader)] --- # [homo] · _Greek_ · 'same' # [icon] · _Greek_ · 'representation' --- layout: true class: normal ### Data and Code Representation --- ```clojure user=> (+ 1 2) 3 ``` ### * We add two numbers ### * But also: ### * A list of the forms +, 1 and 2. --- ```clojure user=> (def time-till-coffee-break (+ (* 5 30) (rand-int 10))) 64 ``` ### * We define the time left until the coffee break ### * But also: ### * A list of the forms def, time-till-coffee-break, and `(+ (* 5 30) (rand-int 10))` --- class: normal # EDN: Extensible Data Notation ### The set of types Clojure knows ```clojure * integers 1, 2 * strings "ohai" * booleans true false * lists (1 2 3) * vectors [1 2 3] * maps {:a 1 :b 2} ``` --- ## Clojure programs are represented in (a superset of) EDN ## Clojure programs handle data that is (a superset of) EDN --- # Clojure: a data-first language
.right[[James Reeves: 'Code is Data'](https://www.booleanknot.com/blog/2016/12/20/code-is-data.html)] --- layout: true class: normal --- background-image: url(images/hammer.jpg) # \`quote\` and \`eval\`: 2 types of hammers --- layout: true class: normal ### quote and eval # quote: Treat a form as Data --- --- ```clojure user=> (+ 1 5) 6 ``` --- ```clojure user=> (+ 1 5) 6 user=> (quote (+ 1 5)) (+ 1 5) ``` --- ```clojure user=> (+ 1 5) 6 user=> (quote (+ 1 5)) (+ 1 5) user=> '(+ 1 5) (+ 1 5) ``` --- layout: true class: normal ### quote and eval # eval: Treat a form as Code --- ```clojure user=> (+ 1 5) 6 ``` --- ```clojure user=> (+ 1 5) 6 user=> '(+ 1 5) (+ 1 5) ``` --- ```clojure user=> (+ 1 5) 6 user=> '(+ 1 5) (+ 1 5) user=> (eval '(+ 1 5)) 6 ``` --- layout: true class: normal --- # Macro: # A way to extend the language ## by creating new constructs the compiler understands --- # Using Macros ## Macros are very common in everyday Clojure programming. ## e.g. `or`, `and`, `when` --- ## macroexpand: a neat tool ```clojure user=> (if-not (even? 41) "not an even number") "not an even number!" user=> (macroexpand '(if-not (even? 41) "not an even number")) (if (clojure.core/not (even? 41)) "not an even number!" nil) ``` --- ## macroexpand: a neat tool ```clojure user => (def n 42) #'user/n ``` --- ## macroexpand: a neat tool ```clojure user => (def n 42) #'user/n user => (when (even? n) (println "n is an even number!") (println "n is: " n) n) n is an even number! n is: 42 42 ``` --- ## macroexpand: a neat tool ```clojure user=> (macroexpand '(when (even? n) (println "n is an even number!") n) (if (even? n) (do (println "n is an even number!") (println "n is: " n) n)) ``` --- ## macroexpand: a neat tool ```clojure user=> (macroexpand '(when (even? n) (println "n is an even number!") n) (if (even? n) (do (println "n is an even number!") (println "n is: " n) n)) user=> (macroexpand (when (even? n) (println "n is an even number!") (println "n is: " n) n)) n is an even number! n is: 42 42 ``` --- class: normal layout: true ## take aways ➡️️
--- ## ➡️️ Homoiconicity: code is data, data is code --- ## ➡️️ Homoiconicity: code is data, data is code ## ➡️️ Clojure is data-oriented, Homoiconicity reflects that --- ## ➡️️ Homoiconicity: code is data, data is code ## ➡️️ Clojure is data-oriented, Homoiconicity reflects that ## ➡️️ 'quote' and 'eval' are useful tools --- ## ➡️️ Homoiconicity: code is data, data is code ## ➡️️ Clojure is data-oriented, Homoiconicity reflects that ## ➡️️ 'quote' and 'eval' are useful tools ## ➡️️ we can extend the language by writing macros --- layout: true class: normal ### What can we do with data? --- # * Visualise it 👀 --- # * Visualise it 👀 # * Analyse it 📈 --- # * Visualise it 👀 # * Analyse it 📈 # * Generate it 💧 --- layout: true class: normal ### Since Clojure is homoiconic, we can ... --- # * Visualise our code 👀 --- # * Visualise our code 👀 # * Analyse our code 📈 --- # * Visualise our code 👀 # * Analyse our code 📈 # * Generate more code! 💧 --- layout: false class: main, middle, center # Part 1: Visualise yourself ## Teaching a program to visualise stats about itself! ### Code is at github.com/vsmart/homoiconicity-talk
#### .left[ \#dzy] --- layout: true class: normal ### Part 1: Visualise yourself --- .center[] --- ## * Find a simple metric --- ## * Find a simple metric ## * Number of defs in a namespace --- ## * Find a simple metric ## * Number of defs in a namespace ## * Visualise it!
.bottom[.left[#### \#dzy]] --- .center[] --- ```clojure user=> (require 'visualise-code.core) nil ``` --- ```clojure user=> (require 'visualise-code.core) nil user=> (in-ns 'visualise-code.core) #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] ``` --- ```clojure user=> (require 'visualise-code.core) nil user=> (in-ns 'visualise-code.core) #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] visualise-code.core=> *ns* #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] ``` --- ```clojure user=> (require 'visualise-code.core) nil user=> (in-ns 'visualise-code.core) #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] visualise-code.core=> *ns* #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] visualise-code.core=> (str *ns*) "visualise-code.core" ``` --- ```clojure user=> (require 'visualise-code.core) nil user=> (in-ns 'visualise-code.core) #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] visualise-code.core=> *ns* #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] visualise-code.core=> (str *ns*) "visualise-code.core" visualise-code.core=> (ns-publics *ns*) {draw-status-bar #'visualise-code.core/draw-status-bar, draw-state #'visualise-code.core/draw-state, update-state #'visualise-code.core/update-state, visualise-code #'visualise-code.core/visualise-code, draw-many-circles #'visualise-code.core/draw-many-circles, setup #'visualise-code.core/setup, current-ns #'visualise-code.core/current-ns, draw-circle #'visualise-code.core/draw-circle} ``` --- ```clojure user=> (require 'visualise-code.core) nil user=> (in-ns 'visualise-code.core) #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] visualise-code.core=> *ns* #object[clojure.lang.Namespace 0xcbb9c04 "visualise-code.core"] visualise-code.core=> (str *ns*) "visualise-code.core" visualise-code.core=> (ns-publics *ns*) {draw-status-bar #'visualise-code.core/draw-status-bar, draw-state #'visualise-code.core/draw-state, update-state #'visualise-code.core/update-state, visualise-code #'visualise-code.core/visualise-code, draw-many-circles #'visualise-code.core/draw-many-circles, setup #'visualise-code.core/setup, current-ns #'visualise-code.core/current-ns, draw-circle #'visualise-code.core/draw-circle} visualise-code.core=> (keys (ns-publics *ns*)) (draw-status-bar draw-state update-state visualise-code draw-many-circles setup current-ns draw-circle) ``` --- ```clojure (def current-ns *ns*) ``` --- ```clojure (def current-ns *ns*) (defn update-state [state] (assoc state :fn-count (count (keys (ns-publics current-ns))))) ``` --- ```clojure (def current-ns *ns*) (defn update-state [state] (assoc state :fn-count (count (keys (ns-publics current-ns))))) (defn draw-background [state] (let [fn-count (:fn-count state) h (mod (* 18 fn-count) 360) s 160 b 200] (q/background (q/color h s b)))) ``` --- ## Other interesting functions for namespaces include 'ns-refers', 'ns-aliases', 'ns-interns' --- class: normal layout: true ## take aways ➡️️
--- --- ## ➡️️ It's easy for a Clojure program to read Clojure programs, even itself! --- ## ➡️️ It's easy for a Clojure program to read Clojure programs, even itself! ## ➡️️ Once you have program data, visualising it is trivial! --- class: main, middle, center layout: false # Part 2: Dev tooling and Testing --- class: normal layout: true ## Part 2: Dev Tooling and Testing --- ## _One thing I really appreciate with code as data in a language such as Clojure is the power of your editor. The power of an editor is related to how well it can interact and analyse code._
## .right[[Joseph Wilk](http://blogs.mulesoft.com/dev/news-dev/code-is-data-data-is-code/)] --- # Why we ❤️️ Clojure --- # Why we ❤️️ Clojure ## The flexibility of the repl --- # Why we ❤️️ Clojure ## The flexibility of the repl ## Powerful editors with inline evaluation --- # Why we ❤️️ Clojure ## The flexibility of the repl ## Powerful editors with inline evaluation ## Live coding! --- layout: true class: normal --- # "Who tests our tests?" --- # Mutation Testing .center[] --- # Mutation Testing .center[] ## __Mutating your code__ in order to __test the completeness of your test suite__ --- # Examples of mutations ## * switch around operators e.g. +,- ```clojure (< x 10) => (> x 10) (+ x 10) => (- x 10) ``` --- # Examples of mutations ## * if => if-not ```clojure (if (= a b) ...) => (if-not (= a b) ...) ``` --- # Examples of mutations ## * delete entire forms ```clojure (defn do-some-maths [a b] (* a b 3)) (defn do-some-maths [a b] ()) ``` --- class: center, middle # Mutant: Mutation testing library in Clojure ## By @jstepien ## Code is on Github [@jstepien/mutant](https://github.com/jstepien/mutant) --- # Generating code :check: like data easily! ```clojure (def ^:private gt-gte (swapping-mutation '<= '<)) (def ^:private lt-lte (swapping-mutation '>= '>)) ``` --- # Generating code :check: like data easily! ```clojure (def ^:private and-or (swapping-mutation 'and 'or)) (def ^:private empty?-seq (swapping-mutation 'empty? 'seq)) ``` --- # Generating code :check: like data easily! ```clojure (def ^:private eq-noteq (swapping-mutation '= 'not=)) ``` --- layout: true class: normal ## take aways ➡️️
--- ## ➡️️ the repl ; powerful editors ; live coding +++ --- ## ➡️️ the repl ; powerful editors ; live coding +++ ## ➡️️ Homoiconicity is the base for awesome dev tooling in Clojure! --- ## ➡️️ the repl ; powerful editors ; live coding +++ ## ➡️️ Homoiconicity is the base for awesome dev tooling in Clojure! ## ➡️️ Mutation testing: generating code, like data, is simple! --- layout: false class: main, middle, center # Part 3: The TweeGeeMee Bot by Roger Allen ### Try it out at twitter.com/tweegeemee ### Code is at github.com/rogerallen/tweegeemee --- class: normal ### Part 3: The TweeGeeMee Bot by @rogerallen .center[] --- class: normal ### Part 3: The TweeGeeMee Bot by @rogerallen ## Inspecting one entity ```clojure { :name "170218_023156_M.clj" :parents ["170217_203119_M.clj"] :hash -1253854099 :image-hash 1645688389 :code (clisk.live/adjust-hsl (clisk.live/x (clisk.live/normalize (clisk.live/vcos (clisk.live/t [-0.7439 0.5524 1.017 -0.2864])))) (clisk.live/v- (clisk.live/vfrac (clisk.live/adjust-hsl (clisk.live/x (clisk.live/normalize (clisk.live/vcos (clisk.live/t [-0.7439 0.5524 1.017 -0.2864])))) (clisk.live/scale (clisk.live/vmod (clisk.live/min-component [2.0579 1.6987 -2.8491 -2.6116]) [2.6967 1.7232]) clisk.live/pos))) clisk.live/pos)) } ``` --- class: normal ### Part 3: The TweeGeeMee Bot by @rogerallen ## Reproducing the image ```clojure => (def code '(clisk.live/vdivide (clisk.live/vfrac (clisk.live/vmin (clisk.live/v+ (clisk.live/alpha clisk.live/grain) [-0.9438 0.4027 2.3753 1.7962]) (clisk.live/sigmoid (clisk.live/vfloor [0.4416 -2.6627 -1.6566])))) (clisk.live/gradient (clisk.live/square (clisk.live/v- [-0.2226 -2.2105 -2.7124 -1.7799] clisk.live/vsnoise))))) ``` --- class: normal ### Part 3: The TweeGeeMee Bot by @rogerallen ## Reproducing the image ```clojure => (def code '(clisk.live/vdivide (clisk.live/vfrac (clisk.live/vmin (clisk.live/v+ (clisk.live/alpha clisk.live/grain) [-0.9438 0.4027 2.3753 1.7962]) (clisk.live/sigmoid (clisk.live/vfloor [0.4416 -2.6627 -1.6566])))) (clisk.live/gradient (clisk.live/square (clisk.live/v- [-0.2226 -2.2105 -2.7124 -1.7799] clisk.live/vsnoise))))) => (show (eval code) :width 720 :height 720) ``` --- class: normal ### Part 3: The TweeGeeMee Bot by @rogerallen ## Reproducing the image ```clojure => (def code '(clisk.live/vdivide (clisk.live/vfrac (clisk.live/vmin (clisk.live/v+ (clisk.live/alpha clisk.live/grain) [-0.9438 0.4027 2.3753 1.7962]) (clisk.live/sigmoid (clisk.live/vfloor [0.4416 -2.6627 -1.6566])))) (clisk.live/gradient (clisk.live/square (clisk.live/v- [-0.2226 -2.2105 -2.7124 -1.7799] clisk.live/vsnoise))))) => (show (eval code) :width 720 :height 720) ``` .center[] --- class: normal layout: true ## take aways ➡️️
--- ## ➡️️ We can share, store, reproduce results easily - since code is data --- ## ➡️️ We can share, store, reproduce results easily - since code is data ## ➡️️ We can generate code - like data - through generative programming! --- ## ➡️️ We can share, store, reproduce results easily - since code is data ## ➡️️ We can generate code - like data - through generative programming! ## ➡️️ Generative programming is great for generative art --- layout: true class: normal --- # clojure.spec --- # clojure.spec ## * Describe data using specs --- # clojure.spec ## * Describe data using specs ## * Validate data against specs --- # clojure.spec ## * Describe data using specs ## * Validate data against specs ## * Generate data from specs --- # clojure.spec ## * Describe data using specs ## * Validate data against specs ## * Generate data from specs --- layout: false class: main, center, middle # Part 4: Self-healing Code by Carin Meier / @gigasquid ### Recorded at [EuroClojure 2016](https://www.youtube.com/watch?v=xvk-Gnydn54) ### Code is at github.com/gigasquid/self-healing --- layout: true class: normal ### Part 4: Self-healing Code by @gigasquid --- # Codephage - Horizontal Code Transfer Systems ## _"...fixes software errors by transferring correct code from a set of donor applications"_
.right[From [Genetic programming with clojure.spec](https://www.youtube.com/watch?v=xvk-Gnydn54)] --- ```clojure user=> (calc-average [0 1 2 3 4]) 2 user=> (calc-average [2 3 4]) 3 ``` --- ```clojure user=> (calc-average [0 1 2 3 4]) 2 user=> (calc-average [2 3 4]) 3 user=> (calc-average []) EXCEPTION!! ``` --- # The Self-healing process ## 1. Recognise a failure has occured ## 2. Detect where the failure comes from. ## 3. Pick the best patch from a given selection ## 4. Replace the failing function with the patch --- class: normal ```clojure user=> (healing/with-healing (calc-average [2 3 4])) 3 user=> (healing/with-healing (calc-average [])) 0 ``` ## Success! --- class: normal layout: true ## take aways ➡️️
--- ## ➡️️ Clojure is a great language for creating terrifying robots. --- ## ~~➡️️ Clojure is a great language for creating terrifying robots.~~ ## ➡️️ Clojure.spec emphasizes and enhances what homoiconicity gives us! --- class: normal layout: true ## Homoiconicity in Clojure gives us: --- --- ## * Extending our language with macros and DSLs --- ## * Extending our language with macros and DSLs ## * Analyse code => powerful development tools --- ## * Extending our language with macros and DSLs ## * Analyse code => powerful development tools ## * Generate code => mutation testing, generative programming --- ## * Extending our language with macros and DSLs ## * Analyse code => powerful development tools ## * Generate code => mutation testing, generative programming ## * Visualise code => different analysis, new tools for devs? --- ## * Extending our language with macros and DSLs ## * Analyse code => powerful development tools ## * Generate code => mutation testing, generative programming ## * Visualise code => different analysis, new tools for devs? # * Many more cool things! --- layout: false class: main, center, middle # Homoiconicity # is # ⚡⚡⚡ Clojure's superpower!! ⚡⚡⚡ --- class: center, middle, main # thank you 💖 ### Slides are at [franka.tech/talks/homoiconicity-talk](https://franka.tech/talks/homoiconicity-talk) ### Find me on [franka.tech](https://franka.tech) or on twitter [@franschm](https://twitter.com/franschm)