Snowpipe

A pipe continuously loads files into a table as they land in a stage. Its meter is unlike anything else in Snowflake: serverless credits charged per GB loaded, with no warehouse involved. File shape no longer changes the credit charge, but it still decides how efficiently that data loads.

Bytes are the unit of billing, files the unit of work

Snowpipe now charges a fixed credit amount per GB loaded: text formats such as CSV and JSON on uncompressed size, binary formats such as Parquet on observed size. The former model added a per-1,000-files fee, which is where the advice that tiny files are expensive comes from. Under per-GB billing, 10,000 one-kilobyte files cost about the same in credits as a single 10 MB file carrying the same rows, but per-file overhead makes them load less efficiently. Event-driven producers that flush a tiny object every few seconds are still the pattern to fix, and batching upstream toward Snowflake’s recommended 100 to 250 MB compressed is usually the entire fix, with no Snowflake-side tuning required.

A meter that scales with noise

A pipe attached to a chatty, low-value feed converts every upstream event into spend, regardless of whether anyone ever queries the target table. Unlike a warehouse there is nothing to suspend: ingestion cost is controlled at the source, by deciding which feeds deserve continuous loading at all and how much data they send.

Discovery and attribution

ZopNight lists pipes with SHOW PIPES IN ACCOUNT, parenting each to its database and skipping pipes inside Snowflake-provided shares. Snowpipe is one of the four serverless credit streams priced within account cost, so ingestion charges never hide inside an undifferentiated total where nobody owns them.

Small-file symptoms show in the history

The ratio of files to bytes is the diagnostic. Healthy batch ingestion shows few files and many bytes per file; the failure signature is the reverse. An enormous file count moving very little data means per-file load overhead dominates, the pattern ZopNight’s small-files rule flags.

Trace pipe spend to its files

SELECT pipe_name,
       SUM(credits_used)   AS credits_30d,
       SUM(files_inserted) AS files_30d,
       SUM(bytes_inserted) AS bytes_30d
FROM SNOWFLAKE.ACCOUNT_USAGE.PIPE_USAGE_HISTORY
WHERE start_time > DATEADD('day', -30, CURRENT_TIMESTAMP())
GROUP BY pipe_name
ORDER BY credits_30d DESC;

Rows here can lag live ingestion by 45 minutes to 3 hours. Divide bytes by files per pipe: a small quotient on a big credit number is the upstream-batching conversation, ready-made.