You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2016-10-19-current-environment.md
+25-5Lines changed: 25 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,16 +12,25 @@ Note: some libraries are mentioned more than once, because they provide function
12
12
## Visualization
13
13
14
14
-**Chart**[](https://github.com/timbod7/haskell-chart){:.github} [](https://hackage.haskell.org/package/Chart)[](http://stackage.org/lts-11/package/Chart)[](http://stackage.org/nightly/package/Chart) : A library for generating 2D Charts and Plots, with backends provided by Cairo (http://hackage.haskell.org/package/Chart-cairo) and Diagrams (http://hackage.haskell.org/package/Chart-diagrams). Documentation: https://github.com/timbod7/haskell-chart/wiki.
15
-
-**plotlyhs**[](https://github.com/diffusionkinetics/open/tree/master/plotlyhs){:.github} [](https://hackage.haskell.org/package/plotlyhs)[](http://stackage.org/lts-11/package/plotlyhs)[](http://stackage.org/nightly/package/plotlyhs) : This is a library for generating JSON value to use with the Plotly.js library. The interface directly reflects the structure of the Plotly.js library and is therefore quite low-level. Lenses are used throughout to set Maybe fields in records to provide both data and configuration options.
15
+
-**plotlyhs**[](https://github.com/diffusionkinetics/open/tree/master/plotlyhs){:.github} [](https://hackage.haskell.org/package/plotlyhs)[](http://stackage.org/lts-11/package/plotlyhs)[](http://stackage.org/nightly/package/plotlyhs) : This is a library for generating JSON value to use with the Plotly.js library. The interface directly reflects the structure of the Plotly.js library and is therefore quite low-level. Lenses are used throughout to set Maybe fields in records to provide both data and configuration options.
16
16
This library does not attempt to communicate with the Plotly API in any other way. All generated plots can be hosted on stand-alone web pages.
17
-
-**hvega**[](https://hackage.haskell.org/package/hvega)[](http://stackage.org/lts-11/package/hvega)[](http://stackage.org/nightly/package/hvega): Support the creation of [Vega-Lite](https://vega.github.io/vega-lite/) visualizations in Haskell.
-**hvega**[](https://hackage.haskell.org/package/hvega)[](http://stackage.org/lts-11/package/hvega)[](http://stackage.org/nightly/package/hvega): Support the creation of [Vega-Lite](https://vega.github.io/vega-lite/) visualizations in Haskell.
19
+
20
+
## Publication
21
+
22
+
-**knit**[](https://hackage.haskell.org/package/knit-haskell)[](http://stackage.org/lts-11/package/knit-haskell)[](http://stackage.org/nightly/package/knit-haskell): knit-haskell is a beginning attempt at bringing some of the benefits of Rmarkdown to Haskell. It includes an effects stack (using polysemy rather than mtl) which includes logging, randomness (via random-fu), a simplified interface to Pandoc and various writer-like effects to intersperse document building with regular code. Various helper functions are provided to simplify common operations, making it especially straightforward to build an HTML document from bits of markdown, latex and Lucid or Blaze html. Support is also included for including hvega visualizations.
23
+
-**pandoc-pyplot**[](https://hackage.haskell.org/package/pandoc-pyplot)[](http://stackage.org/lts-11/package/pandoc-pyplot)[](http://stackage.org/nightly/package/pandoc-pyplot): A Pandoc filter to include figures generated from Python code blocks. Keep the document and Python code in the same location. Output from Matplotlib is captured and included as a figure.
24
+
18
25
19
26
## Data structures
20
27
21
28
### Data frames
22
29
23
-
-**frames**[](https://hackage.haskell.org/package/Frames)[](http://stackage.org/lts-11/package/Frames)[](http://stackage.org/nightly/package/Frames) : User-friendly, type safe, runtime efficient tooling for working with tabular data deserialized from comma-separated values (CSV) files. The type of each row of data is inferred from data, which can then be streamed from disk, or worked with in memory. Also see the comprehensive [tutorial](https://acowley.github.io/Frames/)
24
-
-**analyze**[](https://github.com/ejconlon/analyze){:.github} [](https://hackage.haskell.org/package/analyze)[](http://stackage.org/lts-11/package/analyze)[](http://stackage.org/nightly/package/analyze) : `pandas`-like dataframe operations for tabular data with CSV interface.
30
+
-**Frames**[](https://hackage.haskell.org/package/Frames)[](http://stackage.org/lts-11/package/Frames)[](http://stackage.org/nightly/package/Frames) : User-friendly, type safe, runtime efficient tooling for working with tabular data deserialized from comma-separated values (CSV) files. The type of each row of data is inferred from data, which can then be streamed from disk, or worked with in memory. Also see the comprehensive [tutorial](https://acowley.github.io/Frames/)
31
+
-**Frames-map-reduce**[](https://hackage.haskell.org/package/Frames-map-reduce)[](http://stackage.org/lts-11/package/Frames-map-reduce)[](http://stackage.org/nightly/package/Frames-map-reduce) : This library contains some useful functions for using the map-reduce-folds package with Frames (containers of data rows) from the Frames package. Included, in Frames.MapReduce, are helpers for filtering Frames, splitting records into key and data columns and reattaching key columns after reducing.
32
+
Also included, in the Frames.Folds module, are some helpful functions for building folds of Frames from folds over each column, specified either individually or via a constraint on all the columns being folded over.
33
+
-**analyze**[](https://github.com/DataHaskell/dh-core){:.github} [](https://hackage.haskell.org/package/analyze)[](http://stackage.org/lts-11/package/analyze)[](http://stackage.org/nightly/package/analyze) : `pandas`-like dataframe operations for tabular data with CSV interface.
25
34
Currently maintained within the scope of the [DataHaskell `dh-core` project](https://github.com/datahaskell/dh-core).
26
35
-**bookkeeper**[](https://hackage.haskell.org/package/bookkeeper)[](http://stackage.org/lts-11/package/bookkeeper)[](http://stackage.org/nightly/package/bookkeeper) : A new take on datatypes and records using `OverloadedLabels` (which is available since GHC 8). It bears some similarities to Nikita Volkov's `record` library, but requires no Template Haskell.
27
36
-**colonnade**[](https://github.com/andrewthad/colonnade){:.github} [](https://hackage.haskell.org/package/colonnade)[](http://stackage.org/lts-11/package/colonnade)[](http://stackage.org/nightly/package/colonnade) : The colonnade package provides a way to talk about columnar encodings and decodings of data. This package provides very general types and does not provide a way for the end-user to actually apply the columnar encodings they build to data. Most users will also want to one a companion packages that provides (1) a content type and (2) functions for feeding data into a columnar encoding:
@@ -31,7 +40,10 @@ Currently maintained within the scope of the [DataHaskell `dh-core` project](htt
31
40
* yesod-colonnade for yesod widgets
32
41
* siphon for encoding and decoding CSVs
33
42
43
+
### Streaming / Folds
34
44
45
+
-**map-reduce-folds**[](https://hackage.haskell.org/package/map-reduce-folds)[](http://stackage.org/lts-11/package/map-reduce-folds)[](http://stackage.org/nightly/package/map-reduce-folds) : map-reduce-folds is an attempt to find a good balance between simplicity, performance, and flexibility for simple map/reduce style operations on a foldable container f of some type x (e.g., [x]). The goal of the package is to provide an efficient fold for the computation via the types in the foldl package.
46
+
Folds can be composed Applicatively, which makes it simple to do many such operations on the same data and loop over it only once.
35
47
36
48
37
49
@@ -369,7 +381,15 @@ The datasets package defines two different kinds of datasets:
369
381
### Apache Spark bindings
370
382
371
383
-**sparkle**[](https://hackage.haskell.org/package/sparkle)[](http://stackage.org/lts-11/package/sparkle)[](http://stackage.org/nightly/package/sparkle) : A library for writing resilient analytics applications in Haskell that scale to thousands of nodes, using Spark and the rest of the Apache ecosystem under the hood.
372
-
See the [blog post](https://www.tweag.io/posts/2016-02-25-hello-sparkle.html) for details:
384
+
See the [blog post](https://www.tweag.io/posts/2016-02-25-hello-sparkle.html) for details.
385
+
386
+
-**distributed-dataset**[](https://hackage.haskell.org/package/distributed-dataset)[](http://stackage.org/lts-11/package/distributed-dataset)[](http://stackage.org/nightly/package/distributed-dataset) : A distributed data processing framework in pure Haskell. Inspired by Apache Spark.
387
+
`Control.Distributed.Dataset` provides a type which lets you express transformations on a distributed multiset. Its API is highly inspired by Apache Spark.
388
+
It uses pluggable ShuffleStore's for storing intermediate compuation results. See 'distributed-dataset-aws' for an implementation using S3.
389
+
`Control.Distributed.Fork` contains a fork function which lets you run arbitrary IO actions on remote machines; leveraging StaticPointers language extension and distributed-closure library.
390
+
This module is useful when your task is embarrassingly parallel:
391
+
It uses pluggable Backends for spawning executors. See 'distributed-dataset-aws' for an implementation using AWS Lambda .
392
+
373
393
-**kraps-h**[](https://github.com/krapsh/kraps-haskell){:.github} [](https://hackage.haskell.org/package/kraps-h)[](http://stackage.org/lts-11/package/kraps-h)[](http://stackage.org/nightly/package/kraps-h) : Haskell bindings to Apache Spark. The library consists of:
374
394
- A specification to describe data pipelines in a language-agnostic manner, and a communication protocol to submit these pipelines to Spark.
375
395
- A serving library, called krapsh-server, that implements this specification on top of Spark. It is written in Scala and is loaded as a standard Spark package.
0 commit comments