Skip to content

Commit cd155ed

Browse files
committed
update readme to mention ACIC challenge
1 parent ddd9fea commit cd155ed

File tree

1 file changed

+31
-13
lines changed

1 file changed

+31
-13
lines changed

README.md

Lines changed: 31 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -11,19 +11,37 @@ Framework for evaluating causal inference methods.
1111
- [Authors](#authors)
1212

1313
## General
14-
Causality-Benchmark is a library developed by IBM Research for benchmarking algorithms that
15-
estimate causal effect.
16-
The framework includes unlabeled data, labeled data, and code for scoring algorithm predictions.
17-
It can benchmark predictions of both population effect size and individual effect size.
18-
19-
The evaluation script is not bounded to the provided data, and can be used on other data as
20-
long as some basic requirements are kept regarding the formats.
21-
For more technical details about the evaluation metrics and the data, please refer to the
22-
framework menuscript **TODO: Link to the menuscript/technical report**
23-
24-
Please note that due to GitHub limitation, only a sample of the data is available in this
25-
repository. However, you can manually access and download the entire dataset from the
26-
[Synapse sharing platform](https://www.synapse.org/#!Synapse:syn11294478/files/)
14+
Causality-Benchmark is a library developed by IBM Research Haifa for
15+
benchmarking algorithms that estimate the causal effect of a treatment on
16+
some outcome. The framework includes unlabeled data, labeled data, and code
17+
for scoring algorithm predictions. It can benchmark predictions of both
18+
population effect size and individual effect size.
19+
20+
The feature matrix is derived from the
21+
[linked birth and infant death data](https://www.cdc.gov/nchs/nvss/linked-birth.htm),
22+
and the labeled and unlabeled data are based on simulated models of the
23+
treatment assignment, treatment effect, and censoring.
24+
25+
The evaluation script is not bounded to the provided data,
26+
and can be used on other data as
27+
long as some basic requirements are kept regarding the formats.
28+
The evaluation metrics that are provided include the RMSE, the
29+
effect-normalized RMSE, and several metrics regarding the confidence
30+
intervals.
31+
More technical details regarding the calculated metrics and the formats of the
32+
labeled and unlabeled data will be published soon through a related manuscript.
33+
Most of the details can be found in the
34+
[Casual Inference Cahllenge 2018](https://www.synapse.org/#!Synapse:syn11294478)
35+
website.
36+
37+
Please note that due to GitHub limitation, only a sample of the data is
38+
available in this repository. However, you can manually access and download
39+
the entire dataset from the
40+
[Synapse sharing platform](https://www.synapse.org/#!Synapse:syn11294478/files/).
41+
Furthermore, since the benchmarking tool is used in the
42+
[Casual Inference Cahllenge 2018](https://www.synapse.org/#!Synapse:syn11294478),
43+
the dataset currently includes a handful of example data with labeles.
44+
The full set will be available when the challenge ends.
2745

2846
## Getting Started
2947
### Prerequisites

0 commit comments

Comments
 (0)