Tools, Formats and Protocols
Current tooling
| Purpose | Primary Tool |
|---|---|
| Code formatting and style | black, flake8 and isort |
| CI/CD | GitHub Actions |
| pipeline and data version control | dvc |
| documentation | sphinx and GitHub Pages |
| task execution | invoke |
| exploratory data analisys | pandas and pandas-profiling |
| dashboarding | dash |
| version controlled visualization | altair |
| infrastructure as code | docker and docker-compose |
Serialization
| Data | Format |
|---|---|
| tabular | parquet |
| unstructured | json |
| configuration | yaml |
| metadata | xml |
| python model | joblib |
| documentation | markdown |
Under consideration
- make
- w3c tabluar data model and metadata standards
- PostGIS
- dask
- owlready2
- Julia AST
- Pluto.jl
- commitlint
- libera chat
- superset
- kaggle dataset hosting
- paperswithcode
- mypy