Data Schema

Databases require a configuration file which specifies the data schema - the type and structure of the data, as well as any customized settings. If a data structure changes, the configuration must be modified accordingly.
Configuration File
Hyperspace requires the configuration to be provided in a standard .json format
  • Data configuration is provided under a key named ‘configuration’
  • Lists, sets and dicts must be declared under a key named “'struct_type'”.
  • Vectors are defined under a designated key, including dimension and similarity metric.
  • Fields that are used for cardinality aggregation are defined as “cardinality_field”.
  • low_cardinality, high_cardinality, cardinality_field fields should include proper boolean key in description.
  • Settings can be included under the key ‘settings’.
{'configuration':{‘series’: {'type': 'keyword'},  
                         ‘age_group’: {'type': 'keyword', ‘cardinality_field’: True},  
                         ‘genres': {'struct_type': 'list', 'type': 'keyword', ‘low_cardinality’: True},  
                         'id': {'type': 'integer'},  
                         ‘text embedding': {'dim': 1024, 'metric': 'IP', 'type': 'dense_vector'},  
                         'production_companies': {'struct_type': 'list', 'type': 'keyword'}, 
                         'production_countries': {'struct_type': 'list', 'type': 'keyword'},  
                         'rating': {'type': 'float'},
                         ‘spoken_languages': {'struct_type': 'list', 'type': 'keyword'}, 
                         ‘title': {'type': 'keyword'}
'settings': {‘delimiter’:”’”}