Advance Streamlit Functions #2: Cache, Singleton, Memo
What Is Cache and Why Is It Used?
As I mentioned in my previous Streamlit advance function article, In Streamlit, every browser tab is defined as a session and for every interaction in Streamlit, it reruns your script. That means all of your computations in your script run again and again. However, there could be some. Your Python script may contain time-consuming operations such as complex computation functions, database connections, and model-loading functions. Moreover, you may want to execute your database connection or data-loading function at the beginning of your script, and after that use it again and again without re-execute. For such cases, cache functions are used in Streamlit. To simplify the cache functions, you can think of cache functions as a feature that memorizes the outputs after checking your functions' specific properties.
Usage
Let’s create a simple and interactive app in Streamlit. In this simple app, there are two number input boxes to get float numbers and one button to trigger the summation function. Our main focus in the code snippet should be on the summation function. Because cache works on this function.
Let’s calculate two plus two. When you press the sum button the first time, execution waits 3 seconds, and after that, the sum function is executed. However, when you press the sum button again, execution does not wait and the sum function result is printed instantly. This is because when you press the button for the first time, the cache decorator saves the result for those parameters for the aggregate function. When it is called again, instead of running the function, it calls the result in its “memory”.
Now let’s add three and one. When we click the collect button again, the function will run after waiting three seconds and when you press the button again, the cache decorator will call the result for the relevant input values without running the function. However, when you want to add two and two again, you will see that the result comes from the cache. This means that as you continue your session, the cache keeps different results for different input values entered in the local cache, and when you call those input values again at any time, it can directly call the result of the relevant function from the local cache.
For Streamlit to save a function’s output to the local cache, it needs to check four things.
- The input parameters for your function which wanted to cache
- The value of any external variable used in the function
- The body of your function
- The body of any function used inside the cached function
If all of these conditions are the same as the previous cache record, execution is skipped and the result is gathered from the local cache. If even one differs, the execution of the function is ensured again.
To test these situations, let’s write an inner function inside the aggregation function in our example. Let the task of this internal function be to print numbers. Any changes you make to this body of the inner function or the body of the sum function will cause the sum function to run again.
Singleton and Memo
There are two new primitive cache functions. These are experimental_singleton and experimental_memo. A question may come to your mind: why should I use these instead of using the cache function? The answer is simplicity and speed. For example, according to Streamlit docs, experimental_memo is 10x faster than the classic cache function for large data frames.
Singleton
It can be defined as a key/value store. You can use it for database connections and non-data objects such as Keras sessions. It returns items by reference. Also, its use is quite simple compared to the cache function. You don’t have to think of hash functions that are used in cache functions and additional parameters like allow_output_mutation. Just add to your code for functions and sit back. It’s that simple!
Memo
Unlike singleton, it cached items by value not by reference. You can use it for costly computations, storing data, and dataframe computations. Also, you don’t have to worry about the hash function and accidental mutation while using experimental_memo. It handles them for you. If you ignore unhashable parameters by prefixing them with an underscore
Example usage
You will see two functions in the code above. The first function which is the connect_sql function is connecting to the SQL database. In order not to re-run this connection every time, I put it in the local cache with experimental_singleton. Then I use the other function to pull the necessary information from the SQL database as a dataframe. In the same way, I use experimental_memo to cache the dataframe.
Summary
By using cache functions, you can greatly increase the speed of your Stremalit application. Singleton and memo decorators are still experimental features, but they work very effectively. I used it in my projects and did not encounter any problems. In addition, they are much simpler and faster to use than the classic cache function.
