|
1 | 1 | # LeanDataSdk |
2 | 2 |
|
| 3 | +[](https://github.com/QuantConnect/LeanDataSdk/actions?query=workflow%3A%22Build%20%26%20Test%22) |
| 4 | + |
3 | 5 | ### Getting started |
4 | 6 |
|
5 | 7 | #### Introduction |
6 | 8 |
|
7 | | -The Lean Data SDK is a cross-platform template repository for developing custom data types for Lean. These data types will be consumed by trading algorithms and research environment. |
| 9 | +The Lean Data SDK is a cross-platform template repository for developing custom data types for Lean. |
| 10 | +These data types will be consumed by [QuantConnect](https://www.quantconnect.com/) trading algorithms and research environment, locally or in the cloud. |
8 | 11 |
|
9 | | -It is composed by a .Net solution for the data type and example converter scripts. |
| 12 | +It is composed by example .Net solution for the data type and converter scripts. |
10 | 13 |
|
11 | 14 | #### Prerequisites |
12 | 15 |
|
13 | | -The solution targets dotnet 5, for installation instructions please follow (dotnet)[https://dotnet.microsoft.com/download]. |
| 16 | +The solution targets dotnet 5, for installation instructions please follow [dotnet download](https://dotnet.microsoft.com/download). |
14 | 17 |
|
15 | 18 | The data downloader and converter script can be developed in different ways: Python script, C# or Python jupyter notebook or even a bash script. |
| 19 | +- The python script should be compatible with python 3.6.8 |
| 20 | +- C# notebook will run using [dotnet interactive](https://github.com/dotnet/interactive) |
| 21 | +- Bash script will run on Ubuntu Bionic |
| 22 | + |
| 23 | +Specifically, the enviroment where these scripts will be run is [quantconnect/research](https://hub.docker.com/repository/docker/quantconnect/research) based on [quantconnect/lean:foundation](https://hub.docker.com/repository/docker/quantconnect/lean). |
16 | 24 |
|
17 | 25 | #### Installation |
18 | 26 |
|
19 | 27 | This repository should be forked by each new data provider. |
20 | 28 |
|
21 | | -Once it is cloned locally, should be able to successfully build the solution or execute the conveter scripts. |
| 29 | +Once it is cloned locally, should be able to successfully build the solution, run all tests and execute the conveter scripts. |
| 30 | + |
| 31 | +#### Usage |
| 32 | + |
| 33 | +- Once the repository is forked, the existing example implementation should be adjusted, in the fork, to create a new data type for a particular data set. |
| 34 | +- Converter and downloader scripts should be developed following the [examples in this repository](https://github.com/QuantConnect/LeanDataSdk/tree/master/DataConverterScript). |
| 35 | + |
| 36 | +The script should be provided to `QuantConnect` as well as the fork repository at a particual commit. |
22 | 37 |
|
23 | 38 | ### User guide |
24 | 39 |
|
25 | | -### Tutorial |
| 40 | +### Tutorials |
26 | 41 |
|
27 | 42 | #### Create Data Type |
28 | 43 |
|
29 | 44 | ##### Introduction |
30 | 45 |
|
31 | | -Creating a custom data type will allow Lean algorithms or research environment to consume a particular data set. |
| 46 | +This tutorial we will create a new custom C# data type that will allow Lean algorithms or research environment to consume a particular data set. |
| 47 | + |
| 48 | +##### New Lean Data Type |
| 49 | + |
| 50 | +In [Lean](https://github.com/QuantConnect/Lean) each data type inherits from [BaseData](https://github.com/QuantConnect/Lean/blob/master/Common/Data/BaseData.cs), overrides a set of methods and incoporates any specific property this data set has. |
| 51 | +The `DataLibrary` project holds an example custom data type [MyCustomDataType](https://github.com/QuantConnect/LeanDataSdk/blob/master/DataLibrary/MyCustomDataType.cs). |
| 52 | + |
| 53 | +- `GetSource()` method returns an instance of `SubscriptionDataSource` which will tell Lean from where should it source data for a particular given date, ticker, and configuration. |
| 54 | +- `Reader()` method should return a new instance of this data type for a given line of data |
| 55 | +- `Clone()` Clones the data |
| 56 | +- `RequiresMapping()` Indicates whether the data source is tied to an underlying symbol and requires that corporate events be applied to it as well, such as renames and delistings |
| 57 | +- `IsSparseData()` Indicates whether the data is sparse. If true, we disable logging for missing files |
| 58 | +- `ToString()` converts the instance to string format |
| 59 | +- `DefaultResolution()` gets the default resolution for this data and security type if the user provided none |
| 60 | +- `SupportedResolutions()` gets the supported resolution for this data and security type |
| 61 | +- `DataTimeZone()` specifies the data time zone for this data type |
| 62 | + |
| 63 | +##### Tests |
| 64 | + |
| 65 | +It will be a requisite that each data type has a json and protobuf round trip serialization and deserialization, as well as a clone unit test. Examples provided at [MyCustomDataTypeTests](https://github.com/QuantConnect/LeanDataSdk/blob/master/Tests/MyCustomDataTypeTests.cs) |
32 | 66 |
|
33 | 67 | #### Create Algorithm |
34 | 68 |
|
| 69 | + |
| 70 | + |
35 | 71 | #### Create Data Converters |
36 | 72 |
|
37 | 73 | ##### Python Notebook |
|
0 commit comments