[{"data":1,"prerenderedAt":6},["ShallowReactive",2],{"post-content-postgres-index-only-scan-heap-fetches":3},{"content":4,"lastModified":5},"\u003Cp>The ticket said the order history page felt slow. Not broken, just slow: 40ms on a quiet box, 200ms while the nightly import ran, for sixty rows belonging to one user.\u003C\u002Fp>\n\n\u003Cp>I had already fixed this. Three months earlier I added a covering index, payload columns in INCLUDE, saw the plan say Postgres index-only scan, and closed the ticket.\u003C\u002Fp>\n\n\u003Cp>When I finally opened EXPLAIN again, the plan still said Index Only Scan. That is what cost me an evening. Postgres labels a node Index Only Scan while reading the heap on every row, and the name cannot tell you.\u003C\u002Fp>\n\n\u003Cimg src=\"https:\u002F\u002Fimages.pexels.com\u002Fphotos\u002F6333743\u002Fpexels-photo-6333743.jpeg?auto=compress&cs=tinysrgb&w=1600\" alt=\"Wall of labeled library card catalog drawers, an index you can read without walking to the shelves\" loading=\"lazy\" \u002F>\n\n\u003Ch2>What the plan node is really promising\u003C\u002Fh2>\n\n\u003Cp>A Postgres index entry holds a key and a tuple pointer. Nothing else. No visibility information, because visibility lives in the row header on the heap page, and it changes without the index being touched.\u003C\u002Fp>\n\n\u003Cp>So index-only does not mean \"never touch the heap\". It means \"the index has every column you asked for, so the heap is only needed for visibility\". There is one exemption. Postgres keeps a visibility map with two bits per heap page. The one that matters here is all-visible: every row on that page is visible to every transaction. When it is set, the executor trusts the index entry and skips the heap read.\u003C\u002Fp>\n\n\u003Cp>Two conditions, then: the index covers the query, and the page is marked all-visible. Only the first shows up in the plan node name.\u003C\u002Fp>\n\n\u003Ch2>Five minutes to reproduce it\u003C\u002Fh2>\n\n\u003Cpre>\u003Ccode class=\"language-sql\">CREATE TABLE orders (\n  id bigserial PRIMARY KEY,\n  user_id int NOT NULL,\n  status text NOT NULL,\n  created_at timestamptz NOT NULL,\n  total_cents int NOT NULL,\n  note text\n);\n\nINSERT INTO orders (user_id, status, created_at, total_cents, note)\nSELECT (i % 5000) + 1,\n       (ARRAY['pending','completed','refunded','cancelled'])[1 + i % 4],\n       now() - (i || ' minutes')::interval,\n       1000 + i % 90000,\n       repeat('padding ', 4)\nFROM generate_series(1, 300000) i;\n\n-- 300k rows, padded note column so the heap is realistically wide\nCREATE INDEX idx_orders_user_status\n  ON orders (user_id, status) INCLUDE (created_at, total_cents);\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>Right after the load, with no vacuum yet, the covered query returns 60 rows and the planner picks a Bitmap Heap Scan: Heap Blocks: exact=60, buffers shared hit=60 read=4. No index-only path, because relallvisible was 0 and the planner prices those visits in.\u003C\u002Fp>\n\n\u003Cp>One VACUUM later:\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-sql\">VACUUM (ANALYZE) orders;\n\nEXPLAIN (ANALYZE, BUFFERS)\nSELECT user_id, status, created_at, total_cents\nFROM orders WHERE user_id = 42 AND status = 'completed';\n\n--  Index Only Scan using idx_orders_user_status on orders\n--    Heap Fetches: 0\n--    Buffers: shared hit=5\n--    Execution Time: 0.033 ms\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>Five buffers instead of sixty-four. Same query, same index, one maintenance pass in between. That gap is the whole feature, and it sits at the mercy of vacuum.\u003C\u002Fp>\n\n\u003Cp>Now with write traffic:\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-sql\">UPDATE orders SET note = 'touched' WHERE id % 10 = 0;  -- 30k rows\n\nEXPLAIN (ANALYZE, BUFFERS)\nSELECT user_id, status, created_at, total_cents\nFROM orders WHERE user_id = 42 AND status = 'completed';\n\n--  Index Only Scan using idx_orders_user_status on orders\n--    Heap Fetches: 60\n--    Buffers: shared hit=65\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>Every row came off the heap again. Thirty thousand updates cleared all-visible bits across most of the table: a page loses that bit the moment any row on it changes.\u003C\u002Fp>\n\n\u003Ch2>Where the bits go\u003C\u002Fh2>\n\n\u003Cp>Once a bit is cleared, only VACUUM can put it back, and only on pages where nothing outside its snapshot can see an older row. That is why a long transaction or a stale replication slot stalls coverage cluster-wide: the xmin horizon does not move.\u003C\u002Fp>\n\n\u003Cp>Inserts matter too, and I keep forgetting. New rows land on new pages, and those pages carry no all-visible bit until a vacuum walks them. I inserted 200 rows right after vacuuming the test table and the same query went from Heap Fetches: 0 to 204 out of 260 rows. The newest pages are never covered yet.\u003C\u002Fp>\n\n\u003Cp>Since Postgres 13, autovacuum also fires on inserts, so an append-only table does get its map maintained. Under heavy UPDATE churn the defaults arrive late: two million changed rows on a ten million row table before a worker wakes up.\u003C\u002Fp>\n\n\u003Ch2>The two numbers that do not lie\u003C\u002Fh2>\n\n\u003Cp>First, Heap Fetches from EXPLAIN (ANALYZE, BUFFERS). That is how many heap visits the scan actually made. Compare it with rows returned. Near 1:1 means you paid for the index and got nothing.\u003C\u002Fp>\n\n\u003Cp>Second, relallvisible over relpages in pg_class. A planner estimate, refreshed by VACUUM and ANALYZE, not a live view. Mine read every page all-visible, 3704 of 3704, while the real map had zero pages marked, because it was written before the big UPDATE. That is the number that makes you skip the check.\u003C\u002Fp>\n\n\u003Cp>Ground truth:\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-sql\">CREATE EXTENSION IF NOT EXISTS pg_visibility;\n\nSELECT * FROM pg_visibility_map_summary('orders'::regclass);\n\n all_visible | all_frozen\n-------------+------------\n           0 |          0\n\n-- cluster-wide proxy, no extension needed\nSELECT prs.indexrelname,\n       prs.idx_scan,\n       prs.idx_tup_read,\n       prs.idx_tup_fetch,\n       round(100.0 * prs.idx_tup_fetch \u002F NULLIF(prs.idx_tup_read, 0), 1) AS fetch_pct\nFROM pg_stat_user_indexes prs\nWHERE prs.relname = 'orders'\nORDER BY prs.idx_tup_read DESC;\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>idx_tup_read counts index entries read, idx_tup_fetch counts rows pulled from the heap. I checked both directions on the scratch cluster: 260 read with 0 fetched after a vacuum, then 60 read with 60 fetched after the update. Above a few percent on a query you believed was covered, you have a maintenance problem, not an index problem.\u003C\u002Fp>\n\n\u003Cp>One warning, since it cost me an hour. Posts keep handing out a query that selects heap_fetches from pg_stat_user_indexes. That column does not exist, and the views that come closest call it something else.\u003C\u002Fp>\n\n\u003Ch2>What I actually changed\u003C\u002Fh2>\n\n\u003Cp>Not the index. The index was fine. I set per-table vacuum thresholds on the four hottest tables:\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-sql\">ALTER TABLE orders SET (\n  autovacuum_vacuum_scale_factor = 0.01,\n  autovacuum_vacuum_threshold = 1000,\n  autovacuum_analyze_scale_factor = 0.005\n);\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>One percent of ten million is a hundred thousand changed rows, about twenty minutes of traffic, so coverage recovers while the day is still running. The cost is real background I\u002FO.\u003C\u002Fp>\n\n\u003Cp>I also cut the INCLUDE list from four to two columns. A wider payload means a bigger index, more pages to keep visible, and heavier writes; updating an INCLUDE column writes a new index entry and kills HOT updates for that row.\u003C\u002Fp>\n\n\u003Cp>The one I keep relearning: a query covered last month is not covered now, because someone added a field to the API response and one more column to the SELECT. Adding \u003Ccode>note\u003C\u002Fcode> to my test query flipped it back to a Bitmap Heap Scan. Coverage needs an assertion in CI, not a memory.\u003C\u002Fp>\n\n\u003Cp>The disk side of this is in \u003Ca href=\"\u002Fposts\u002Fpostgres-table-bloat-autovacuum-falling-behind\u002F\">table bloat and autovacuum falling behind\u003C\u002Fa>: same disease, different symptom. If the planner will not touch your index at all, that is \u003Ca href=\"\u002Fposts\u002Fpostgres-not-using-my-index-what-i-check\u002F\">a different checklist\u003C\u002Fa>.\u003C\u002Fp>\n\n\u003Cp>The vacuum settings and both queries live in \u003Ca href=\"\u002Fsnippetark\u002F\">Snippet Ark\u003C\u002Fa>, in a note called postgres coverage, which is where this should have started.\u003C\u002Fp>\n","2026-11-12",1794910420008]