The following best practices are recommended on how to use data sources:
Size: It is important to be mindful that data sources get downloaded by OPA, which stores the contents in memory. Do not upload more data into a data source than you want OPA to load into memory. Currently, 10MB is a good size for data sources, both for OPA and DAS. At 50MB, you may start to see some timeouts in the DAS API and will need to be sure OPA has the memory to load it. If you want to load more than 50MB into a single data source, then contact the Styra support team.
Quantity: Each data source in DAS has some overhead both for the user and DAS. The API must list the data sources attached to each system or stack and fetch the contents out of backend storage; users need to create and populate the contents of each data source. Styra recommends using one 10MB data source instead of ten 1MB data sources.
- All data sources are recommended not to exceed 10-20MB in size.
- If your data is larger, break it into several smaller data sources.
- If your data is smaller, combine them into a single data source.