@@ -11,19 +11,37 @@ Framework for evaluating causal inference methods.
1111 - [ Authors] ( #authors )
1212
1313## General
14- Causality-Benchmark is a library developed by IBM Research for benchmarking algorithms that
15- estimate causal effect.
16- The framework includes unlabeled data, labeled data, and code for scoring algorithm predictions.
17- It can benchmark predictions of both population effect size and individual effect size.
18-
19- The evaluation script is not bounded to the provided data, and can be used on other data as
20- long as some basic requirements are kept regarding the formats.
21- For more technical details about the evaluation metrics and the data, please refer to the
22- framework menuscript ** TODO: Link to the menuscript/technical report**
23-
24- Please note that due to GitHub limitation, only a sample of the data is available in this
25- repository. However, you can manually access and download the entire dataset from the
26- [ Synapse sharing platform] ( https://www.synapse.org/#!Synapse:syn11294478/files/ )
14+ Causality-Benchmark is a library developed by IBM Research Haifa for
15+ benchmarking algorithms that estimate the causal effect of a treatment on
16+ some outcome. The framework includes unlabeled data, labeled data, and code
17+ for scoring algorithm predictions. It can benchmark predictions of both
18+ population effect size and individual effect size.
19+
20+ The feature matrix is derived from the
21+ [ linked birth and infant death data] ( https://www.cdc.gov/nchs/nvss/linked-birth.htm ) ,
22+ and the labeled and unlabeled data are based on simulated models of the
23+ treatment assignment, treatment effect, and censoring.
24+
25+ The evaluation script is not bounded to the provided data,
26+ and can be used on other data as
27+ long as some basic requirements are kept regarding the formats.
28+ The evaluation metrics that are provided include the RMSE, the
29+ effect-normalized RMSE, and several metrics regarding the confidence
30+ intervals.
31+ More technical details regarding the calculated metrics and the formats of the
32+ labeled and unlabeled data will be published soon through a related manuscript.
33+ Most of the details can be found in the
34+ [ Casual Inference Cahllenge 2018] ( https://www.synapse.org/#!Synapse:syn11294478 )
35+ website.
36+
37+ Please note that due to GitHub limitation, only a sample of the data is
38+ available in this repository. However, you can manually access and download
39+ the entire dataset from the
40+ [ Synapse sharing platform] ( https://www.synapse.org/#!Synapse:syn11294478/files/ ) .
41+ Furthermore, since the benchmarking tool is used in the
42+ [ Casual Inference Cahllenge 2018] ( https://www.synapse.org/#!Synapse:syn11294478 ) ,
43+ the dataset currently includes a handful of example data with labeles.
44+ The full set will be available when the challenge ends.
2745
2846## Getting Started
2947### Prerequisites
0 commit comments