Skip to content

Commit 9f6d0b1

Browse files
Add GH workflow. Improve readme
1 parent 1b6f42e commit 9f6d0b1

7 files changed

Lines changed: 70 additions & 6 deletions

File tree

.github/workflows/build.yml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: Build & Test
2+
3+
on:
4+
push:
5+
branches: ['*']
6+
tags: ['*']
7+
pull_request:
8+
branches: [master]
9+
10+
jobs:
11+
build:
12+
runs-on: ubuntu-20.04
13+
container:
14+
image: quantconnect/lean:foundation
15+
steps:
16+
- uses: actions/checkout@v2
17+
18+
- name: Build
19+
run: dotnet build /p:Configuration=Release /v:quiet /p:WarningLevel=1 MyCustomDataType.sln
20+
21+
- name: Run Tests
22+
run: dotnet test ./Tests/bin/Release/net5.0/Tests.dll
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

README.md

Lines changed: 42 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,73 @@
11
# LeanDataSdk
22

3+
[![Build Status](https://github.com/QuantConnect/LeanDataSdk/workflows/Build%20%26%20Test/badge.svg)](https://github.com/QuantConnect/LeanDataSdk/actions?query=workflow%3A%22Build%20%26%20Test%22)
4+
35
### Getting started
46

57
#### Introduction
68

7-
The Lean Data SDK is a cross-platform template repository for developing custom data types for Lean. These data types will be consumed by trading algorithms and research environment.
9+
The Lean Data SDK is a cross-platform template repository for developing custom data types for Lean.
10+
These data types will be consumed by [QuantConnect](https://www.quantconnect.com/) trading algorithms and research environment, locally or in the cloud.
811

9-
It is composed by a .Net solution for the data type and example converter scripts.
12+
It is composed by example .Net solution for the data type and converter scripts.
1013

1114
#### Prerequisites
1215

13-
The solution targets dotnet 5, for installation instructions please follow (dotnet)[https://dotnet.microsoft.com/download].
16+
The solution targets dotnet 5, for installation instructions please follow [dotnet download](https://dotnet.microsoft.com/download).
1417

1518
The data downloader and converter script can be developed in different ways: Python script, C# or Python jupyter notebook or even a bash script.
19+
- The python script should be compatible with python 3.6.8
20+
- C# notebook will run using [dotnet interactive](https://github.com/dotnet/interactive)
21+
- Bash script will run on Ubuntu Bionic
22+
23+
Specifically, the enviroment where these scripts will be run is [quantconnect/research](https://hub.docker.com/repository/docker/quantconnect/research) based on [quantconnect/lean:foundation](https://hub.docker.com/repository/docker/quantconnect/lean).
1624

1725
#### Installation
1826

1927
This repository should be forked by each new data provider.
2028

21-
Once it is cloned locally, should be able to successfully build the solution or execute the conveter scripts.
29+
Once it is cloned locally, should be able to successfully build the solution, run all tests and execute the conveter scripts.
30+
31+
#### Usage
32+
33+
- Once the repository is forked, the existing example implementation should be adjusted, in the fork, to create a new data type for a particular data set.
34+
- Converter and downloader scripts should be developed following the [examples in this repository](https://github.com/QuantConnect/LeanDataSdk/tree/master/DataConverterScript).
35+
36+
The script should be provided to `QuantConnect` as well as the fork repository at a particual commit.
2237

2338
### User guide
2439

25-
### Tutorial
40+
### Tutorials
2641

2742
#### Create Data Type
2843

2944
##### Introduction
3045

31-
Creating a custom data type will allow Lean algorithms or research environment to consume a particular data set.
46+
This tutorial we will create a new custom C# data type that will allow Lean algorithms or research environment to consume a particular data set.
47+
48+
##### New Lean Data Type
49+
50+
In [Lean](https://github.com/QuantConnect/Lean) each data type inherits from [BaseData](https://github.com/QuantConnect/Lean/blob/master/Common/Data/BaseData.cs), overrides a set of methods and incoporates any specific property this data set has.
51+
The `DataLibrary` project holds an example custom data type [MyCustomDataType](https://github.com/QuantConnect/LeanDataSdk/blob/master/DataLibrary/MyCustomDataType.cs).
52+
53+
- `GetSource()` method returns an instance of `SubscriptionDataSource` which will tell Lean from where should it source data for a particular given date, ticker, and configuration.
54+
- `Reader()` method should return a new instance of this data type for a given line of data
55+
- `Clone()` Clones the data
56+
- `RequiresMapping()` Indicates whether the data source is tied to an underlying symbol and requires that corporate events be applied to it as well, such as renames and delistings
57+
- `IsSparseData()` Indicates whether the data is sparse. If true, we disable logging for missing files
58+
- `ToString()` converts the instance to string format
59+
- `DefaultResolution()` gets the default resolution for this data and security type if the user provided none
60+
- `SupportedResolutions()` gets the supported resolution for this data and security type
61+
- `DataTimeZone()` specifies the data time zone for this data type
62+
63+
##### Tests
64+
65+
It will be a requisite that each data type has a json and protobuf round trip serialization and deserialization, as well as a clone unit test. Examples provided at [MyCustomDataTypeTests](https://github.com/QuantConnect/LeanDataSdk/blob/master/Tests/MyCustomDataTypeTests.cs)
3266

3367
#### Create Algorithm
3468

69+
70+
3571
#### Create Data Converters
3672

3773
##### Python Notebook

Tests/MyCustomDataTypeTests.cs

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,5 +32,11 @@ public void ProtobufRoundTrip()
3232
{
3333

3434
}
35+
36+
[Test]
37+
public void Clone()
38+
{
39+
40+
}
3541
}
3642
}

0 commit comments

Comments
 (0)