Elasticsearch, how to store data per date?

Question

I'd like to store daily statistics of sales.

By daily, I mean day of year.

Conceptually the data would look like the following.

{ 
  name: bmw,
  daily_statics : {
     1: { sold_count: 5},
     2: { sold_count: 3},

  ...
     365: {sold_count: 2}
}

I'm trying to create a mapping in elasticsearch-dsl-py and having trouble.

I guess it would be similarly troublesome to create a mapping for this document?

• edit

I'd like to view seasonal trend of data. so day-of-year 1 and day-of-year 365 is actually pretty close input point. Besides, I only need one year worth of data

question from:https://stackoverflow.com/questions/65936853/elasticsearch-how-to-store-data-per-date

Answer 1

You can let ES generate the mappings for you with the help of dynamic template path matching:

PUT myindex
{
  "mappings": {
    "dynamic_templates": [
      {
        "catch_daily_statistics": {
          "path_match": "daily_statistics.*",
          "mapping": {
            "type": "object",
            "properties": {
              "sold_count": {
                "type": "long"
              }
            }
          }
        }
      }
    ]
  }
}

But before you proceed with this architectural choice, I'd recommend to first check this answer which deals with the difficulties of aggregating on dicts-of-dicts.

It'd be more reasonable to use daily indices as @Evaldas pointed out. But I'd go with nested fields of the form:

POST myindex/_doc
{
  "name": "bmw",
  "daily_stats": [
    {
      "day_date": "2021/01/01",
      "day_iso_num": 1,
      "sold_count": 5
    },
    { ... }
  ]
}

That way, you'll be able to perform

• range queries on day_date
• range and term queries on day_iso_num (since day-of-year is different year-over-year...)
• and any metric aggs (among others) on sold_count