我只想保留 5 天的数据，如果少于保留日期，则删除其余数据。需要记住每天生成 2000000 行

Question

DELIMITER $$
CREATE PROCEDURE sp_delete_data()
BEGIN
  DECLARE i INT DEFAULT 0;
  DECLARE loop_counter INT DEFAULT 0;
  DECLARE retain_days datetime;
  DECLARE delete_days datetime;
  SET loop_counter=(SELECT ROUND(count(*)/100,0) FROM data2 WHERE datetime<(SELECT  DATE_ADD(min(datetime),INTERVAL 1 DAY) FROM data2));
  SET retain_days=(SELECT DATE_SUB(now(),INTERVAL 5 DAY)); -- 5 days data will keep
  SET delete_days =(SELECT  DATE_ADD(min(datetime),INTERVAL 1 DAY) FROM data2); -- check old data from table data2
        WHILE i <= loop_counter DO
            IF retain_days>delete_days THEN
                DELETE FROM data2 where datetime<delete_days LIMIT 1000;
            END IF;
            SET i = i + 1;
        END WHILE;  
END$$
DELIMITER ;

I want to keep only 5 days data and delete rest of the data if it is less than the retention date. Since each day data generate almost 2000000 rows that is why it is difficult to delete whole data by one shot. That is why I want to delete 100000 data in each loop. Here, loop_counter variable used to find how many loop we should use for this day data. retain_days variable define to find retention date delete_days variable define to find deleted date. based on retain_days and delete_days varoable data will retain and delete. Fnally this procedure will call by event every 1 day.

我的代码循环没有按预期工作。需要高手解答。如果删除这样的数据有任何性能问题，请告诉我。提前致谢

Answer 1

只需创建一个运行每天一次的活动：

CREATE EVENT purge_old_data
    ON SCHEDULE EVERY '1' DAY
    STARTS CURRENT_TIMESTAMP()
    ON COMPLETION PRESERVE
    COMMENT 'Delete rows older than 5 days'
    DO 
BEGIN
  DELETE
    FROM data2
  WHERE `datetime` < DATE_SUB(CURRENT_TIMESTAMP(), INTERVAL 5 DAY);
END;

您不应使用循环来删除较小的块。在 SQL 数据库循环实际上使性能变差。如果您必须在第一个运行中删除数百或数百万行，这对 MariaDB 来说真的不是问题。

Answer 2

PARTITION BY RANGE 每个分区都是 2 小时的数据。然后 DROP PARTITION 将非常快速丢弃数据 -- 多比 DELETE.

更多关于使用分区的信息：http://mysql.rjweb.org/doc.php/partitionmaint

备选方案：http://mysql.rjweb.org/doc.php/deletebig

特别是，第二个 link 展示了如何通过 PRIMARY KEY 'continually' 运行遍历数据，一次删除 1000 行。完成后重复。

注意：以下是有问题的：

DELETE FROM data2
   where datetime<delete_days LIMIT 1000;

如果没有 INDEX(datetime)，它将花费大量时间寻找要删除的行。这样的索引，还有在索引和数据之间来回跳动1000次的开销。在任何一种情况下，都必须将 1000 行放入重做日志中，以防发生崩溃。

我只想保留 5 天的数据，如果少于保留日期，则删除其余数据。需要记住每天生成 2000000 行

I want to keep only 5 days data and delete rest of the data if it is less than the retention date. Need to remember each day 2000000 row generate

mysql

sql

stored-procedures

mariadb