A database in Python? You're kidding...
TinyOlap is an open-source, multi-dimensional, in-memory, model-first OLAP engine written in plain Python. As an in-process Python library, it empowers developers to build lightweight solutions for extended planning & analysis (xP&A) and many other numerical use cases.
TinyOlap is also quite handy as a smart alternative to Pandas DataFrames when your data is multi-dimensional, requires hierarchical aggregations or complex calculations.
TinyOlap aims to honour and mimic great commercial products like IBM TM/1, Jedox PALO, Infor d/EPM or SAP SAC. If their scalability and performance is not required, or their technical complexity or cost is not reasonable for your purpose, then TinyOlap might be for you. TinyOlap's database capabilities are on par with commercial products, almost!
Please be advised that TinyOlap is still an alpha version and under active development.
Your feedback, ideas and wishes are very welcome!
Will TinyOlap meet my requirements?
Feature-wise definitely, performance-wise it will depend. You'll need to try!
Some performance figures (Macbook Air M1, single thread, Python 3.10, 6-dimensional cube):
1M records require ∅900MB in RAM and 45MB on disk, write data with ∅100k up to 250k cells/sec,
read data with ∅25k up to 250k cells/sec, max. internal throughput up to ∅25m aggregations/sec,
report rendering with ∅50 up to 150 reports/sec, depending on the number of cells in the report
and the overall data model complexity (esp. the use of leaf-level rules).
TinyOlap is perfectly suited for use cases with 5k up to 500k cells/database. Above 1m, maybe 5m cells/database TinyOlap might leave the fun zone. You'll need to try!
Here's a screenshot from the provided samples/enterprise_web_demo. The database mimics the reporting, planning & forecasting of a fictitious but real-world-sized international company called 'Tiny Corp.'. As the entire database is just generated by code, you can play around with the number of legal entities and products and check if TinyOlap will meet your requirements.
Show me some code!
Ok! Here´s how Elon Musk is doing his business planning for Tesla - allegedly ;-)
This example shows all the necessary steps to create and use a TinyOlap database. First you need to define some dimensions and a cube that will store your data. If required, you can add your own business logic, called 'rules'. Then you're ready to add or import some data and analyse or process your data. A simple console based reporting is also provided as well. Enjoy...
import random from tinyolap.cell import Cell from tinyolap.decorators import rule from tinyolap.database import Database from tinyolap.view import View @rule(cube="sales", trigger=["Delta %"]) def delta_percent(c: Cell): if c.Plan: # prevent potential division by zero return c.Delta / c.Plan return None def elons_random_numbers(low: float = 1000.0, high: float = 2000.0): return random.uniform(low, high) # Purpose: Support Elon Musk on his business planning & reporting for Tesla def tesla_business_planning(console_output: bool = True): # 1st - define an appropriate 5-dimensional cube (the data space) db = Database("tesla") cube = db.add_cube("sales", [ db.add_dimension("datatypes").edit() .add_many(["Actual", "Plan"]) .add_many("Delta", ["Actual", "Plan"], [1.0, -1.0]) .add_many("Delta %") .commit(), db.add_dimension("years").edit().add_many( ["2021", "2022", "2023"]).commit(), db.add_dimension("periods").edit().add_many( "Year", ["Q1", "Q2", "Q3", "Q4"]).commit(), db.add_dimension("regions").edit().add_many( "Total", ["North", "South", "West", "East"]).commit(), db.add_dimension("products").edit().add_many( "Total", ["Model S", "Model 3", "Model X", "Model Y"]).commit() ]) # 2nd - (if required) add custom business logic, so called 'rules'. # Register the rule that has been implemented above. Take a look. cube.register_rule(delta_percent) # 3rd - (optional) some beautifying, set number formats db.dimensions["datatypes"].members["Delta"].format = "{:+,.0f}" db.dimensions["datatypes"].members["Delta %"].format = "{:+.2%}" # 4th - to write data to the cubes, just define and address and assign a value cube["Plan", "2021", "Q1", "North", "Model S"] = 400.0 # write a single value cube["Plan", "2021", "Q1", "North", "Model X"] = 200.0 # write a single value # 5th - TinyOlap's strength is manipulating larger areas of data # That's the Elon Musk way of planning - what a lazy boy ;-) # The next statement will address <<<ALL EXISTING DATA>>> over all years, periods, # regions and products, and set all existing values to 500.0. Currently, there are # only 2 values 400.0 and 200.0 in the cube, so just these will be changed. cube["Plan"] = 500.0 # Let's see if this has worked properly... if cube["Plan", "2021", "Q1", "North", "Model S"] != 500.00: raise ValueError("TinyOlap is cheating...") # Elon might be lazier than expected... # The 'True' arg in the following statement will force writing the number 500.0 # to <<<REALLY ALL>>> years, periods, regions and products combinations at once. cube["Plan"].set_value(500.0, True) # 3 x 4 x 4 x 4 = all 192 values := 500.0 # For 2023 Elon is planning to skyrocket: 50% more for 2023 cube["Plan", "2023"] = cube["Plan", "2022"] * 1.50 # Now it's time for 'Actual' data # What??? Elon probably wants to take a shortcut here... # He simply hands in a Python function to generate all the 'Actual' data. cube["Actual"].set_value(elons_random_numbers, True) # Where already done! Our first TinyOlap database is ready to use. # 6th - reading data and simple reporting if console_output: # let's create a minimal default report and dump it to the console print(View(cube).refresh().to_console_output()) # finally, let's congratulate Elon dev_percent = cube["Delta %", "2023", "Year", "Total", "Total"] print(f"\nTesla's is {dev_percent:+.2%} above 'Plan' for 2023. " f"Congratulations, Elon!") return db if __name__ == "__main__": tesla_business_planning()
Pssst! The round button below is our share button.