From 105264b5cdf16f310f23345b468063e97768672f Mon Sep 17 00:00:00 2001 From: "A. Sophie Blee-Goldman" Date: Fri, 24 May 2019 10:35:16 -0700 Subject: [PATCH] MINOR: Updated configuration docs with RocksDBConfigSetter#close (#6784) The old docs here used a now deprecated method to set the block cache size. In switching over to the new one we would now need to construct a Cache object and therefore also need to close it, so this is a good opportunity to demonstrate the RocksDBConfigSetter#close method that will need to be implemented by users. Reviewers: Guozhang Wang --- .../developer-guide/config-streams.html | 23 +++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/docs/streams/developer-guide/config-streams.html b/docs/streams/developer-guide/config-streams.html index 6e8b711500e..8e7f950ca58 100644 --- a/docs/streams/developer-guide/config-streams.html +++ b/docs/streams/developer-guide/config-streams.html @@ -684,19 +684,32 @@

Here is an example that adjusts the memory size consumed by RocksDB.

    public static class CustomRocksDBConfig implements RocksDBConfigSetter {
 
+         // These objects should be member variables so they can be closed in RocksDBConfigSetter#close.
+       private org.rocksdb.Cache cache = new org.rocksdb.LRUCache(16 * 1024L * 1024L);
+       private org.rocksdb.Filter filter = new org.rocksdb.BloomFilter();
+
        @Override
        public void setConfig(final String storeName, final Options options, final Map<String, Object> configs) {
          // See #1 below.
          BlockBasedTableConfig tableConfig = new org.rocksdb.BlockBasedTableConfig();
-         tableConfig.setBlockCacheSize(16 * 1024 * 1024L);
+         tableConfig.setBlockCache(cache);
          // See #2 below.
          tableConfig.setBlockSize(16 * 1024L);
          // See #3 below.
          tableConfig.setCacheIndexAndFilterBlocks(true);
+       // See #4 below.
+         tableConfig.setFilter(filter);
          options.setTableFormatConfig(tableConfig);
-         // See #4 below.
+         // See #5 below.
          options.setMaxWriteBufferNumber(2);
        }
+
+       @Override
+       public void close(final String storeName, final Options options) {
+         // See #6 below.
+         cache.close();
+         filter.close();
+       }
     }
 
 Properties streamsSettings = new Properties();
@@ -706,10 +719,12 @@
               

                 
Notes for example:

                 
    
-                  
  1. BlockBasedTableConfig tableConfig = new org.rocksdb.BlockBasedTableConfig(); Reduce block cache size from the default, shown here,  as the total number of store RocksDB databases is partitions (40) * segments (3) = 120.
    2. 
-                  
  2. tableConfig.setBlockSize(16 * 1024L); Modify the default block size per these instructions from the RocksDB GitHub.
    3. 
+                  
  3. BlockBasedTableConfig tableConfig = new org.rocksdb.BlockBasedTableConfig(); Reduce block cache size from the default, shown here,  as the total number of store RocksDB databases is partitions (40) * segments (3) = 120.
    4. 
+                  
  4. tableConfig.setBlockSize(16 * 1024L); Modify the default block size per these instructions from the RocksDB GitHub.
    5. 
                   
  5. tableConfig.setCacheIndexAndFilterBlocks(true); Do not let the index and filter blocks grow unbounded. For more information, see the RocksDB GitHub.
    6. 
+                  
  6. tableConfig.setFilter(filter); Creating/setting a new BlockBasedTableConfig overwrites the BloomFilter Streams uses by default. This is an important optimization and you should consider (re)setting it if you construct your own tableConfig. See  RocksDB Bloom Filter docs for details.
    7. 
                   
  7. options.setMaxWriteBufferNumber(2); See the advanced options in the RocksDB GitHub.
    8. 
+                  
  8. cache.close(); You must implement close to free the memory of any objects that extend org.rocksdb.RocksObject that you constructed. See  RocksJava docs for more details.
    9.