In meiner Anwendung habe ich eine Abfrage, die eine Suche in der Tabelle "files" durchführt.
Die Tabelle "files" ist durch "f". "Created" partitioniert (siehe Tabellendefinition) und hat ~ 26 Millionen Zeilen für den Client 19 ("f". "Cid = 19).
Der Punkt hier ist, wenn ich diese Abfrage mache:
SELECT "f"."id" AS "FileId"
, "f"."name" AS "FileName"
, "f"."year" AS "Fileyear"
, "f"."cid" AS "clientId"
, "f"."created" AS "FileDate"
, CASE WHEN ("vnVE0"."value" is not null AND "vnVE0"."value" != '')
THEN CAST("vnVE0"."value" AS decimal(28,2))
ELSE 0 END AS "keywordValueCol0_numeric"
FROM files "f"
OUTER APPLY
(
SELECT DISTINCT
VT.[value]
FROM dbo.value_number AS VT
WHERE
VT.id_file = F.id
AND VT.id_field = 260
) AS "vnVE0"
WHERE "grapado" IS NULL AND "masterversion" IS NULL AND ("f"."year" = 2013 OR "f"."year" = 0) AND "f"."cid" = 19
GROUP BY "f"."id", "f"."name", "f"."year", "f"."cid", "f"."created", CASE WHEN ("vnVE0"."value" is not null AND "vnVE0"."value" != '')
THEN CAST("vnVE0"."value" AS decimal(28,2))
ELSE 0 END
ORDER BY (SELECT NULL)
OFFSET 0 ROWS FETCH NEXT 50 ROWS ONLY;
Ich erhalte die Ergebnisse in 0 Sekunden mit folgendem Ausführungsplan: https://www.brentozar.com/pastetheplan/?id=SkV0-FDcG
Aber wenn ich versuche, nach "Name" zu bestellen, wird die Abfrage zu langsam:
SELECT "f"."id" AS "FileId"
, "f"."name" AS "FileName"
, "f"."year" AS "Fileyear"
, "f"."cid" AS "clientId"
, "f"."created" AS "FileDate"
, CASE WHEN ("vnVE0"."value" is not null AND "vnVE0"."value" != '')
THEN CAST("vnVE0"."value" AS decimal(28,2))
ELSE 0 END AS "keywordValueCol0_numeric"
FROM files "f"
OUTER APPLY
(
SELECT DISTINCT
VT.[value]
FROM dbo.value_number AS VT
WHERE
VT.id_file = F.id
AND VT.id_field = 260
) AS "vnVE0"
WHERE "grapado" IS NULL AND "masterversion" IS NULL AND ("f"."year" = 2013 OR "f"."year" = 0) AND "f"."cid" = 19
GROUP BY "f"."id", "f"."name", "f"."year", "f"."cid", "f"."created", CASE WHEN ("vnVE0"."value" is not null AND "vnVE0"."value" != '')
THEN CAST("vnVE0"."value" AS decimal(28,2))
ELSE 0 END
ORDER BY "f"."name"
OFFSET 0 ROWS FETCH NEXT 50 ROWS ONLY;
Diese Abfrage dauert 11 Minuten, um mir die Ergebnisse mit dem folgenden Ausführungsplan zurückzugeben: https://www.brentozar.com/pastetheplan/?id=Sk3Fbtv9M
Wenn ich die Reihenfolge nach Spalte ändere, ist das Ergebnis dasselbe.
Wie Sie im Ausführungsplan sehen können, kostet der Index "files_mv" 61%. Dies ist die Indexdefinition:
CREATE NONCLUSTERED INDEX [files_mv] ON [dbo].[files]
(
[masterversion] ASC,
[year] ASC,
[cat_id] ASC,
[cid] ASC,
[eid] ASC,
[grapado] ASC,
[sub_id] ASC,
[tip_id] ASC
)
INCLUDE ( [id],
[name]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
GO
Ich verwende SQL Server mit Azure. Insbesondere Azure SQL-Datenbank mit Preis- / Modellstufe "S4 Estándar (200 DTUs)".
Ich erhalte viele Daten, aber ich denke, dass die Internetverbindung kein Engpass ist, weil ich bei anderen Abfragen auch viele Daten erhalte und sie schneller sind.
Außerdem füge ich Daten in diese Tabellen ein, und in wenigen Tagen werden bis zu 240 Millionen Zeilen in der Dateitabelle (für eine CID) und 480 Millionen Zeilen in der Tabelle value_number vorhanden sein.
zusätzliche Information
Partitionsfunktion "PF_files_partitioning":
CREATE PARTITION FUNCTION PF_files_partitioning (DATETIME2(7))
AS
RANGE LEFT FOR VALUES ( '2013-03-31 23:59:59',
'2013-06-30 23:59:59',
'2013-09-30 23:59:59',
'2013-12-31 23:59:59',
'2014-03-31 23:59:59',
'2014-06-30 23:59:59',
'2014-09-30 23:59:59',
'2014-12-31 23:59:59',
'2015-03-31 23:59:59',
'2015-06-30 23:59:59',
'2015-09-30 23:59:59',
'2015-12-31 23:59:59',
'2016-03-31 23:59:59',
'2016-06-30 23:59:59',
'2016-09-30 23:59:59',
'2016-12-31 23:59:59',
'2017-03-31 23:59:59',
'2017-06-30 23:59:59',
'2017-09-30 23:59:59',
'2017-12-31 23:59:59',
'2018-03-31 23:59:59')
Partitionsschema "PS_files_partitioning":
CREATE PARTITION SCHEME PS_files_partitioning AS PARTITION PF_files_partitioning ALL TO ([PRIMARY]);
** Ich werde ungefähr 15 Millionen Zeilen in jeder Partition haben.
Datei-Tabelle:
CREATE TABLE [dbo].[files](
[id] [bigint] IDENTITY(1,1) NOT NULL,
[cid] [tinyint] NOT NULL,
[eid] [bigint] NOT NULL,
[cat_id] [bigint] NOT NULL,
[tip_id] [bigint] NULL,
[sub_id] [bigint] NULL,
[year] [smallint] NOT NULL,
[caducidad] [smallint] NULL,
[grapadopri] [int] NOT NULL,
[grapado] [bigint] NULL,
[name] [nvarchar](255) NOT NULL,
[extension] [tinyint] NOT NULL,
[size] [bigint] NOT NULL,
[id_doc] [bit] NOT NULL,
[observaciones] [nvarchar](255) NOT NULL,
[indexed] [bit] NOT NULL,
[signed] [bit] NOT NULL,
[created] [datetime2](7) NOT NULL,
[name_lower] [nvarchar](255) NOT NULL,
[modified] [datetime2](7) NULL,
[related] [bit] NOT NULL,
[masterversion] [bigint] NULL,
[versioned] [bit] NOT NULL,
[hwsignature] [tinyint] NOT NULL,
[blockedUserId] [smallint] NULL,
CONSTRAINT [PK_files_id] PRIMARY KEY CLUSTERED
(
[id] ASC,
[created] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created]),
CONSTRAINT [files$estructure_unique] UNIQUE NONCLUSTERED
(
[cat_id] ASC,
[tip_id] ASC,
[sub_id] ASC,
[year] ASC,
[name] ASC,
[grapado] ASC,
[created] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
)
value_number Tabelle:
CREATE TABLE [dbo].[value_number](
[id] [bigint] IDENTITY(1,1) NOT NULL,
[id_file] [bigint] NOT NULL DEFAULT ((0)),
[id_field] [bigint] NOT NULL DEFAULT ((0)),
[value] [nvarchar](255) NULL DEFAULT (NULL),
[id_doc] [bigint] NULL DEFAULT (NULL)
CONSTRAINT [PK_value_number_id] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
)
Indexe der Dateitabelle
CREATE NONCLUSTERED INDEX [files_clientes] ON [dbo].[files]
(
[cid] ASC
)
INCLUDE ([id]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [files_grapado] ON [dbo].[files]
(
[grapado] ASC
)
INCLUDE ( [id],
[name]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [files_mv] ON [dbo].[files]
(
[masterversion] ASC,
[year] ASC,
[cat_id] ASC,
[cid] ASC,
[eid] ASC,
[grapado] ASC,
[sub_id] ASC,
[tip_id] ASC
)
INCLUDE ( [id],
[name]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [files_ocr] ON [dbo].[files]
(
[cid] ASC,
[grapado] ASC,
[indexed] ASC,
[masterversion] ASC,
[extension] ASC
)
INCLUDE ( [id],
[eid],
[cat_id],
[tip_id],
[sub_id],
[year],
[name]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [files_ocr2] ON [dbo].[files]
(
[cid] ASC,
[eid] ASC,
[grapado] ASC,
[indexed] ASC,
[masterversion] ASC,
[extension] ASC
)
INCLUDE ( [id],
[cat_id],
[tip_id],
[sub_id],
[year],
[name]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [files_ocr3] ON [dbo].[files]
(
[cid] ASC,
[cat_id] ASC,
[grapado] ASC,
[indexed] ASC,
[masterversion] ASC,
[extension] ASC
)
INCLUDE ( [eid],
[tip_id],
[sub_id],
[year],
[name]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [busqueda_name] ON [dbo].[files]
(
[cid] ASC,
[eid] ASC,
[grapado] ASC,
[year] ASC
)
INCLUDE ( [id],
[cat_id],
[tip_id],
[sub_id],
[grapadopri],
[name],
[size],
[id_doc],
[signed],
[created],
[modified],
[related],
[masterversion]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [busqueda2] ON [dbo].[files]
(
[cid] ASC,
[eid] ASC,
[cat_id] ASC,
[grapado] ASC,
[masterversion] ASC,
[year] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [cid] ON [dbo].[files]
(
[cid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [eid] ON [dbo].[files]
(
[eid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [extension] ON [dbo].[files]
(
[extension] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [FK_files_archivo] ON [dbo].[files]
(
[grapado] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [FK_files_tipo] ON [dbo].[files]
(
[tip_id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [grapadopri] ON [dbo].[files]
(
[grapadopri] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [index_all] ON [dbo].[files]
(
[cid] ASC,
[eid] ASC,
[grapado] ASC,
[masterversion] ASC
)
INCLUDE ( [cat_id],
[tip_id],
[sub_id],
[year],
[grapadopri],
[name],
[size],
[id_doc],
[signed],
[created],
[modified],
[related],
[versioned]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [missing_index_7_6] ON [dbo].[files]
(
[cid] ASC,
[eid] ASC,
[grapado] ASC,
[name] ASC,
[year] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [ocrCloudClients] ON [dbo].[files]
(
[grapado] ASC,
[indexed] ASC,
[extension] ASC
)
INCLUDE ( [cid],
[eid],
[cat_id],
[tip_id],
[sub_id]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [searchEntity] ON [dbo].[files]
(
[cid] ASC,
[eid] ASC,
[grapado] ASC,
[masterversion] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
CREATE NONCLUSTERED INDEX [sub_id] ON [dbo].[files]
(
[sub_id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON PS_files_partitioning([created])
Indizes der Tabelle value_number
CREATE NONCLUSTERED INDEX [searchValues] ON [dbo].[value_number]
(
[id_field] ASC
)
INCLUDE ( [id_file],
[value]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
CREATE NONCLUSTERED INDEX [search] ON [dbo].[value_number]
(
[id_file] ASC,
[id_field] ASC
)
INCLUDE ( [value]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
CREATE NONCLUSTERED INDEX [id_field] ON [dbo].[value_number]
(
[id_field] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
CREATE NONCLUSTERED INDEX [FK_valueesN_documento] ON [dbo].[value_number]
(
[id_doc] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
CREATE NONCLUSTERED INDEX [FK_valueesN_archivo] ON [dbo].[value_number]
(
[id_file] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
Statistiken sind aktuell. Ich habe die Datentypen für das Jahr und andere Spalten geändert und jetzt scheint die Leistung etwas besser zu sein, aber der Ausführungsplan ist immer noch der gleiche. Ich versuche, die Kardinalitätsschätzung (Indexänderung) zu korrigieren, hatte aber noch keinen Erfolg. Laut Azure-Dokumentation sollte die Datenbank eine Kompatibilitätsstufe von 130 aufweisen, und mit ProductVersion 12.0 habe ich bereits eine Kompatibilitätsstufe von 100.