{
  "_meta": {
    "schema": "https://www.drawdecisiontree.com/decision-dag.schema.json",
    "source": "https://www.drawdecisiontree.com",
    "description": "DrawDecisionTree.com is a free tool for building, sharing, and embedding interactive decision trees. This file is the machine-readable export of a published decision tree. The `dsl` field contains the original source in the Decision DAG DSL; the `dag` schema is documented at the URL in `schema` above.",
    "links": {
      "interactive": "https://www.drawdecisiontree.com/t/ma-operating-system/practical-data-how-many-layers-do-i-need.html",
      "embed": "https://www.drawdecisiontree.com/embed/path/ma-operating-system/practical-data-how-many-layers-do-i-need",
      "dsl_reference": "https://www.drawdecisiontree.com/decision-tree-dsl-reference.html",
      "guides": "https://www.drawdecisiontree.com/guides",
      "schema_docs": "https://www.drawdecisiontree.com/decision-dag.schema.json",
      "author_trees": "https://www.drawdecisiontree.com/trees/ma-operating-system"
    },
    "generated_at": "2026-05-29T12:05:39.243Z"
  },
  "author": {
    "handle": "ma-operating-system",
    "first_name": "M&AOperatingSystem",
    "last_name": null,
    "avatar_url": "4d26d57f-9751-4220-beae-98346f806aa8/avatar-1777319673762.svg",
    "display_name": "M&AOperatingSystem"
  },
  "file": {
    "id": "57c66e97-3d79-43c5-a3b8-ed586427d96a",
    "name": "Practical Data: How many Layers Do I need?",
    "public_slug": "practical-data-how-many-layers-do-i-need",
    "updated_at": "2026-05-12T10:37:14.298069+00:00",
    "url": "https://www.drawdecisiontree.com/t/ma-operating-system/practical-data-how-many-layers-do-i-need.html",
    "json_url": "https://www.drawdecisiontree.com/t/ma-operating-system/practical-data-how-many-layers-do-i-need/tree.json",
    "dsl_url": "https://www.drawdecisiontree.com/t/ma-operating-system/practical-data-how-many-layers-do-i-need/tree.dag"
  },
  "meta": {
    "description": "This decision tree is aimed at Chief Data Officers (CDO) and  Senior Technical Architects trying to figure out the right medallian style architecturedata warehouse product manager responsible for figuring out the most appropriate mechanism for integrating new data sources to a four-tier data warehouse, use this decision tree to identify the most effectivel integration strategy possible.   The tool presumes three architecture principles - i) all data consumption takes place from the final data products layer and ii) The further to the \"right\" the integration takes place towards platinum, the more efficient the technology data processing (ie less storage and less compute) iii) Finally we prefer to materialise data as views over adding addition storage layers.",
    "mode": "elimination",
    "entry": "Q1",
    "tags": [],
    "image": "https://www.maoperatingsystem.com/hubfs/medalion-layers.png"
  },
  "questions": [
    {
      "id": "Q1",
      "text": "Does Data arrive as flat files of either encrypted or unencrypted data?"
    },
    {
      "id": "A",
      "text": "YES - If data arrives as files, you will need an initial blob-storage landing zone before data can be consumed as records. [COPPER]"
    },
    {
      "id": "B",
      "text": "NO - If no data arrives as bulk files, you can skip copper."
    },
    {
      "id": "Q2",
      "text": "What is the content of the Data to be consumed in the warehouse?"
    },
    {
      "id": "A",
      "text": "ENTITY: The data contains information about a specific entity or object that follows a specific  lifecycle over time and will ultimately need to be included as part of a target logical data model [COPPER, BRONZE-M, BRONZE-S, SILVER-M, SILVER-S, GOLD-M, GOLD-S, PLATINUM-M]"
    },
    {
      "id": "B",
      "text": "TRANSACTIONAL: The data contains point-in-time information about a specific event that needs to be included in a target logical data model, either as a list of events or as summarized data.[ COPPER, BRONZE-M, BRONZE-S, SILVER-M, SILVER-S, GOLD-M, GOLD-S, PLATINUM-M]"
    },
    {
      "id": "C",
      "text": "UNDERLYING: The data contains low-level transactional information about a specific event that will ultimately be aggregated or analyzed in some way before derived data is included in the target data model. [COPPER, BRONZE-M, BRONZE-S]"
    },
    {
      "id": "Q3",
      "text": "What is the technical integration type for obtaining the data from the source?"
    },
    {
      "id": "A",
      "text": "FILES: Data is pulled or pushed from the source system as files and needs to \"land\" somewhere before being consumed into structured data records [COPPER]"
    },
    {
      "id": "B",
      "text": "DATABASE: Data is pulled directly to a database warehouse, where data can either be queried or triggers established to represent data changes via SQL or other structured query language. [BRONZE-M, BRONZE-S, SILVER-M, SILVER-S]"
    },
    {
      "id": "C",
      "text": "API: Data is provided via a dedicated API endpoint that can be polled or pushed via a webhook to get one or a few records at a time [BRONZE-S, SILVER-S]"
    },
    {
      "id": "D",
      "text": "MESSAGES: Data is received via a real-time message queue as whole or partial records. [BRONZE-S, SILVER-S, GOLD-S]"
    },
    {
      "id": "Q4",
      "text": "Are the files readable as they arrive, or do they require unencrypting or unpackaging before you can understand the data?"
    },
    {
      "id": "A",
      "text": "READABLE: No additional processing is required and the files can be processed as is into the warehouse [COPPER]"
    },
    {
      "id": "B",
      "text": "ENCODED:  We need to apply a specific decoding tool before we can process the data into the warehouse [COPPER]"
    },
    {
      "id": "Q5",
      "text": "What data model, structure, and values does the data arrive in?"
    },
    {
      "id": "A",
      "text": "SOURCE MODEL: The data arrives in a proprietary model with source-specific values that need to be converted into a target model before being used. [COPPER, BRONZE-M, BRONZE-S]"
    },
    {
      "id": "B",
      "text": "TARGET MODEL: The data is already formatted to conform to the target logical model and does not need any transformation before it can be used.[COPPER, BRONZE-M, BRONZE-S, SILVER-M, SILVER-S,PLATINUM-M]"
    },
    {
      "id": "Q6",
      "text": "Is the source the SOLE source of all information for a specific data subject/object, or will it need to be combined with other sources to create a complete logical model?"
    },
    {
      "id": "A",
      "text": "SOLE-SOURCE: The data source already conforms to the target logical model AND is the ONLY source of data for this data subject. No other data consolidation is needed to establish the complete data records. [COPPER, BRONZE-S, BRONZE-M, SILVER-S, SILVER-M, GOLD-M, GOLD-S, PLATINUM-M]"
    },
    {
      "id": "B",
      "text": "PARTIAL-SOURCE: While the data source already conforms to the target logical data model, it does not contain everything we need about these records, nor does it contain the full population, and it will need to be consolidated with other sources before being used. [COPPER, BRONZE-S, BRONZE-M, SILVER-S, SILVER-M]"
    },
    {
      "id": "Q7",
      "text": "What additional special situations do you need to take into account when considering the integration strategy?"
    },
    {
      "id": "A",
      "text": "NONE: No Additional capabilities are required."
    },
    {
      "id": "B",
      "text": "SOURCE AUDITING: We are required to be able to evidence the exact data content in its original form that arrived from the source system, separate from our internal warehouse control and processing requirements. [COPPER, BRONZE-S, SILVER-S, GOLD-S]"
    },
    {
      "id": "C",
      "text": "HIGH PERFORMANCE: We are required to make the data available immediately after it becomes available from the source to downstream consumers. [GOLD-M, PLATINUM-M]"
    }
  ],
  "outcomes": [
    {
      "id": "COPPER",
      "label": "Integrate to COPPER File Landing Zone"
    }
  ],
  "dsl": "dag: Practical Data: How many Layers Do I need?\nversion: 1.0.0\nimage: https://www.maoperatingsystem.com/hubfs/medalion-layers.png\ndescription: This decision tree is aimed at Chief Data Officers (CDO) and  Senior Technical Architects trying to figure out the right medallian style architecturedata warehouse product manager responsible for figuring out the most appropriate mechanism for integrating new data sources to a four-tier data warehouse, use this decision tree to identify the most effectivel integration strategy possible.   The tool presumes three architecture principles - i) all data consumption takes place from the final data products layer and ii) The further to the \"right\" the integration takes place towards platinum, the more efficient the technology data processing (ie less storage and less compute) iii) Finally we prefer to materialise data as views over adding addition storage layers.\nentry: Q1\nmode: elimination\n\nQ1: Does Data arrive as flat files of either encrypted or unencrypted data?\n  A: YES - If data arrives as files, you will need an initial blob-storage landing zone before data can be consumed as records. [COPPER]\n  B: NO - If no data arrives as bulk files, you can skip copper.\n\nQ2: What is the content of the Data to be consumed in the warehouse?\n  A: ENTITY: The data contains information about a specific entity or object that follows a specific  lifecycle over time and will ultimately need to be included as part of a target logical data model [COPPER, BRONZE-M, BRONZE-S, SILVER-M, SILVER-S, GOLD-M, GOLD-S, PLATINUM-M]\n  B: TRANSACTIONAL: The data contains point-in-time information about a specific event that needs to be included in a target logical data model, either as a list of events or as summarized data.[ COPPER, BRONZE-M, BRONZE-S, SILVER-M, SILVER-S, GOLD-M, GOLD-S, PLATINUM-M]\n  C: UNDERLYING: The data contains low-level transactional information about a specific event that will ultimately be aggregated or analyzed in some way before derived data is included in the target data model. [COPPER, BRONZE-M, BRONZE-S]\n\nQ3: What is the technical integration type for obtaining the data from the source?\n  A: FILES: Data is pulled or pushed from the source system as files and needs to \"land\" somewhere before being consumed into structured data records [COPPER]\n  B: DATABASE: Data is pulled directly to a database warehouse, where data can either be queried or triggers established to represent data changes via SQL or other structured query language. [BRONZE-M, BRONZE-S, SILVER-M, SILVER-S]\n  C: API: Data is provided via a dedicated API endpoint that can be polled or pushed via a webhook to get one or a few records at a time [BRONZE-S, SILVER-S]\n  D: MESSAGES: Data is received via a real-time message queue as whole or partial records. [BRONZE-S, SILVER-S, GOLD-S] \n\nQ4: Are the files readable as they arrive, or do they require unencrypting or unpackaging before you can understand the data? \n  when: Q3=A\n  A: READABLE: No additional processing is required and the files can be processed as is into the warehouse [COPPER]\n  B: ENCODED:  We need to apply a specific decoding tool before we can process the data into the warehouse [COPPER]\n\nQ5: What data model, structure, and values does the data arrive in?\n  A: SOURCE MODEL: The data arrives in a proprietary model with source-specific values that need to be converted into a target model before being used. [COPPER, BRONZE-M, BRONZE-S]\n  B: TARGET MODEL: The data is already formatted to conform to the target logical model and does not need any transformation before it can be used.[COPPER, BRONZE-M, BRONZE-S, SILVER-M, SILVER-S,PLATINUM-M]\n\nQ6: Is the source the SOLE source of all information for a specific data subject/object, or will it need to be combined with other sources to create a complete logical model? \n  when: Q5=B\n  A: SOLE-SOURCE: The data source already conforms to the target logical model AND is the ONLY source of data for this data subject. No other data consolidation is needed to establish the complete data records. [COPPER, BRONZE-S, BRONZE-M, SILVER-S, SILVER-M, GOLD-M, GOLD-S, PLATINUM-M]\n  B: PARTIAL-SOURCE: While the data source already conforms to the target logical data model, it does not contain everything we need about these records, nor does it contain the full population, and it will need to be consolidated with other sources before being used. [COPPER, BRONZE-S, BRONZE-M, SILVER-S, SILVER-M]\n\nQ7: What additional special situations do you need to take into account when considering the integration strategy? \n  A: NONE: No Additional capabilities are required.\n  B: SOURCE AUDITING: We are required to be able to evidence the exact data content in its original form that arrived from the source system, separate from our internal warehouse control and processing requirements. [COPPER, BRONZE-S, SILVER-S, GOLD-S]\n  C: HIGH PERFORMANCE: We are required to make the data available immediately after it becomes available from the source to downstream consumers. [GOLD-M, PLATINUM-M]\n\n[PLATINUM-M]: Integrate to PLATINUM (as a materialized View)\n  color: #E5E4E2\n  description: The available source data and consumers demand speed of access for this specific dataset. The data is available via a standard SQL-like query interface, so materialize it straight into Platinum as its own Data Product.\n\n[GOLD-M]: Integrate to Gold (as a materialized View)\n  color: #D4AF37\n  description: The available source data already conforms to the target schema AND is the sole source of all information associated with a specific target object model AND the data is available via an SQL-style interface that can be queried remotely.  As such there is no need for additional normalization, consolidation or other value-add capabilities. As such we can  integrate this source straight to the gold layer as a materialized view.\n\n[GOLD-S]: Integrate to Gold (as stored data)\n  color: #D4AF37\n  description: The available source data already conforms to the target schema AND is the sole source of all information associated with a specific target object model.  As such, there is no need for additional normalization, consolidation, or other value-add capabilities. As such we can  integrate this source straight to the gold layer.\n\n[SILVER-M]: Integrate to Silver (as Materialized View)\n  color: #C0C0C0\n  description: The available source data already conforms to the target schema and is available via an SQL-like query engine.  As such, establish a materialized view of the data directly into the Silver layer\n\n[SILVER-S]: Integrate to Silver (as stored Data)\n  color: #C0C0C0\n  description: The available source data already conforms to the target schema; however, it is only accessible through integrations that require loading and storing it directly in the warehouse. \n\n[BRONZE-M]: Integrate to BRONZE Layer (As materialized View)\n  color: #CD7F32\n  description: Insert new records into Bronze as they can be read from the source system.  No need to initiate file copies of the in-flight data.  Bronze becomes the auditable history of all records received by the data warehouse. \n\n[BRONZE-S]: Integrate to BRONZE Layer (As stored data)\n  color: #CD7F32\n  description: Insert new records into Bronze as they can be read from the source system.  No need to initiate file copies of the in-flight data.  Bronze becomes the auditable history of all records received by the data warehouse. \n\n[COPPER]: Integrate to COPPER File Landing Zone\n   color: #B87333\n   description: Leverage a Copper file-based landing zone where all incoming files can be stored before processing the data into the bronze stage of the data warehouse.  Inbound files can be stored for an extended period to meet audit requirements or simply purged after a minimum retention period.  Establish loaders that process each incoming file, decrypt or convert the resulting data as needed, and insert new rows into the Bronze layer.\n\n\n\n"
}