feat: merge staging (vector store, doc saver/loader) into main (#23)

* feat(vector_store): Implement RedisVectorStore class

This commit introduces the RedisVectorStore class, extending VectorStore
to enable vector storage and retrieval using Memorystore Redis.

* feat: add integration test for MemorystoreChatMessageHistory (#13)

* fix: fix linter and encoding of MemorystoreChatMessageHistory class. (#20)

* fix: fix linter and encoding of MemorystoreChatMessageHistory class.

* fix: use relative path for __init__ imports

* feat(vector store): Add usage demo and enhance Redis integration (#17)

* feat(vector store): Add usage demo and enhance Redis integration

- Add Jupyter notebook in `docs/` for using langchain vector store with Memorystore Redis.
- Update `vector_store.py` for improved performance and reliability with Redis.
- Modify `__init__.py` to reflect package structure changes and new functionalities.
- Include `requirements.txt` and `setup.py` for easy installation and dependency management.
- Add `state_of_the_union.txt` in `docs/` as a sample dataset for the notebook demo.

* incorporate review feedback

* added missing license header

* excluded the test file from license check

* reformatted source files

* incorporated review feedback

* fixed lint errors

* fixed more lint errors

* removed key_prefix argument from vectorstore

* incorporated review feedback

* fixed formatting errors

* fixed a bad merge

* suppress mypy errors for setuptools

* trying mypy.ini

* remove setup.py and requirements

* add numpy dependency

* Update integration.cloudbuild.yaml

* feat: add MemorystoreDocumentSaver class (#18)

* feat: add MemorystoreDocumentSaver class

* fix: use variable in the worker pool of the Cloud Build integration

* fix: update doc saver for json encoding

* fix: align key_prefix behavior

* fix: add back key_prefix if keys_or_ids is given

* fix: fix indent of the comments

* fix: get encoder directly from the Redis client

* fix: add pipeline support for Redis client

* fix: reorder prefix and content_field in constructor

* fix: insert prefix to all given document IDs

* fix: fix tests prefix and chat encoding

* feat: add MemorystoreDocumentLoader class (#24)

* refactor(vector-store): enhance validation and initialization processes (#22)

* feat(vector store): Add usage demo and enhance Redis integration

- Add Jupyter notebook in `docs/` for using langchain vector store with Memorystore Redis.
- Update `vector_store.py` for improved performance and reliability with Redis.
- Modify `__init__.py` to reflect package structure changes and new functionalities.
- Include `requirements.txt` and `setup.py` for easy installation and dependency management.
- Add `state_of_the_union.txt` in `docs/` as a sample dataset for the notebook demo.

* incorporate review feedback

* added missing license header

* excluded the test file from license check

* reformatted source files

* incorporated review feedback

* fixed lint errors

* fixed more lint errors

* removed key_prefix argument from vectorstore

* incorporated review feedback

* fixed formatting errors

* fixed a bad merge

* suppress mypy errors for setuptools

* trying mypy.ini

* remove setup.py and requirements

* add numpy dependency

* refactor(vector-store): enhance validation and initialization processes

- Add validation for unsupported data types and negative vector sizes in VectorIndexConfig.
- Refactor field_name parameters in HNSWConfig and FLATConfig to support type hinting with optional strings.
- Enhance RedisVectorStore initialization by enforcing type checks for client, index_name, and embedding_service.
- Introduce an optional 'ids' parameter in methods to allow explicit document identifiers, improving document management.
- Adjust method documentation and error messages for clarity and consistency.

* Incorporated review feedback

* fixed formatting

* incorporated review feedback

* formatting

* fixed bugs

* fix formatting

* added vector store unittests

* actualy added vector store tests

* fix formatting

* fix: fix styles and formats

---------

Co-authored-by: Craig Chi <craigchi@google.com>

* fix: rename doc_ to document_ for consistency

* fix: add delete API for document saver and raise error if key exists

* fix: batch delete keys in document saver if specified

---------

Co-authored-by: Ping Xie <pingxie@google.com>
Co-authored-by: Ping Xie <pingxie@outlook.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>

Author: 4 people

.github/header-checker-lint.yml

@@ -1,6 +1,6 @@
 {"allowedCopyrightHolders": ["Google LLC"],
  "allowedLicenses": ["Apache-2.0", "MIT", "BSD-3"],
- "ignoreFiles": ["**/requirements.txt", "**/requirements-test.txt", "**/__init__.py", "samples/**/constraints.txt", "samples/**/constraints-test.txt"],
+ "ignoreFiles": ["**/requirements.txt", "**/requirements-test.txt", "**/__init__.py", "samples/**/constraints.txt", "samples/**/constraints-test.txt", "docs/state_of_the_union.txt"],
  "sourceFileExtensions": [
  	"ts", 
  	"js", 
@@ -12,4 +12,4 @@
  	"html",
  	"txt"
  ]
-}
+}