https://docs.julialang.org/en/v1.0.0/manual/performance-tips/
- Have no global variables
- Add types ot function definitions to have smaller structs
- Add types to containers
- Avoid passing variable number of arguments to a function
- Avoid changing the type of a vriable
- Seperate structure setup and computation. Compiler knows what the types for computations become and can become faster
- Pre allocation data structures in functions would speed things up (I'm not sure this is always true)
- use @. is faster than alternatives for vectorized operations
- Arrays should be read and written contiguously for significant speedups
https://www.youtube.com/watch?v=R81pmvTP_Ik
- Can use CUDA libraries, numerical code up till a keras like library
- Flux ecosystem can allow exports of models to the browser so can play against alpha go in the browser
- Performance of DL libraries is a compiler problem
- Uses Zygote.jl for AD libary
- It's possible to handtune pytorch and more to get speed up to Julia but the Flux compiler is smarter
- Error messages from inside a TF graph will show the whole stack but with Zygote
- Wow: Can inspect LLVM code to see what zygote actually does and it's obvious how fast the code can be
Can build a DSL where you setup optimization constraints and objectives https://github.com/JuliaOpt/JuMP.jl/blob/master/examples/cannery.jl Language really makes you appreciate LISP like expressiveness