kcopyd 提供的是一个易于使用的接口，隐藏了异步：dm-kcopyd.c 重点在于 "d"，即对 kcopyd_jobs 的异步管理，而 copy 相关的具体 async IO 细节在 dm-io.c 中实现
kcopyd provides the ability to copy a range of sectors from one block-device to one or more other block-devices, with an asynchronous completion notification (dm-io 提供了 async/sync IO service，kcopyd 使用了 async IO service).
使用者只需要提供自己的 dm_kcopyd_throttle (可以为 NULL)、src dm_io_region 和 dest dm_io_region(s)、dm_kcopyd_notify_fn，异步由 kcopyd 负责，用户不需要关心
It is used by various dm target types, including mirror, thin, snapshot, cache, writeback, and zoned.
Users of kcopyd must first create a dm_kcopyd_client object. This is done with a call to dm_kcopyd_client_create(). The throttle arg can be NULL if users don't want any throttling.
struct dm_kcopyd_client *dm_kcopyd_client_create(struct dm_kcopyd_throttle *throttle);
To start a copy job, the user must set up dm_io_region structures to describe the source and destinations of the copy. Each dm_io_region indicates a block-device along with the starting sector and size of the region. The source of the copy is given as one dm_io_region structure, and the destinations of the copy are given as an array of dm_io_region structures.
struct dm_io_region {
struct block_device *bdev;
sector_t sector;
sector_t count;
};
To start the copy, the user calls dm_kcopyd_copy(), passing in the client pointer, pointers to the source and destination dm_io_regions, the name of a completion callback routine, and a pointer to some context data for the copy.
int dm_kcopyd_copy(struct dm_kcopyd_client *kc,
struct dm_io_region *from,
unsigned int num_dests,
struct dm_io_region *dests,
unsigned int flags,
dm_kcopyd_notify_fn fn,
void *context);
typedef void (*dm_kcopyd_notify_fn)(int read_err,
unsigned long write_err,
void *context);
When the copy completes, kcopyd will call the user's completion routine, passing back the user's context pointer. It will also indicate if a read or write error occurred during the copy.
When a user is done with all their copy jobs, they should call dm_kcopyd_client_destroy() to delete the kcopyd client, which will release the associated memory pages.
void dm_kcopyd_client_destroy(struct dm_kcopyd_client *kc);
struct dm_kcopyd_throttle
commit df5d2e9089c7d5b8c46f767e4278610ea3e815b9
Author: Mikulas Patocka <mpatocka@redhat.com>
Date: Fri Mar 1 22:45:49 2013 +0000
dm kcopyd: introduce configurable throttling
This patch allows the administrator to reduce the rate at which kcopyd
issues I/O.
Each module that uses kcopyd acquires a throttle parameter that can be
set in /sys/module/*/parameters.
We maintain a history of kcopyd usage by each module in the variables
io_period and total_period in struct dm_kcopyd_throttle. The actual
kcopyd activity is calculated as a percentage of time equal to
"(100 * io_period / total_period)". This is compared with the user-defined
throttle percentage threshold and if it is exceeded, we sleep.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
struct dm_kcopyd_throttle {
unsigned throttle;
unsigned num_io_jobs;
unsigned io_period;
unsigned total_period;
unsigned last_jiffies;
};
/*
* kcopyd clients that want to support throttling must pass an initialised
* dm_kcopyd_throttle struct into dm_kcopyd_client_create().
* Two or more clients may share the same instance of this struct between
* them if they wish to be throttled as a group.
*
* This macro also creates a corresponding module parameter to configure
* the amount of throttling.
*/
#define DECLARE_DM_KCOPYD_THROTTLE_WITH_MODULE_PARM(name, description) \
static struct dm_kcopyd_throttle dm_kcopyd_throttle = { 100, 0, 0, 0, 0 }; \
module_param_named(name, dm_kcopyd_throttle.throttle, uint, 0644); \
MODULE_PARM_DESC(name, description)
struct dm_kcopyd_client
#define SUB_JOB_SIZE 128
#define SPLIT_COUNT 8
#define MIN_JOBS 8
#define RESERVE_PAGES (DIV_ROUND_UP(SUB_JOB_SIZE << SECTOR_SHIFT, PAGE_SIZE))
/*-----------------------------------------------------------------
* Each kcopyd client has its own little pool of preallocated
* pages for kcopyd io.
*---------------------------------------------------------------*/
struct dm_kcopyd_client {
struct page_list *pages;
调用 dm_kcopyd_client_create 的时候会通过
client_reserve_pages 预分配 RESERVE_PAGES
个 pages
unsigned nr_reserved_pages; 初始化为 RESERVE_PAGES
unsigned nr_free_pages; 初始化为 RESERVE_PAGES
struct dm_io_client *io_client;
wait_queue_head_t destroyq;
mempool_t job_pool;
struct workqueue_struct *kcopyd_wq;
alloc_workqueue("kcopyd", WQ_MEM_RECLAIM, 0)
struct work_struct kcopyd_work;
关联 dm-kcopyd.c/do_work
kcopyd does this every time it's woken up
struct dm_kcopyd_throttle *throttle;
atomic_t nr_jobs; 初始化为 0
/*
* We maintain three lists of jobs:
*
* i) jobs waiting for pages
* ii) jobs that have pages, and are waiting for the io to be issued.
* iii) jobs that have completed.
*
* All three of these are protected by job_lock.
*/
spinlock_t job_lock;
struct list_head complete_jobs;
struct list_head io_jobs;
struct list_head pages_jobs;
};
struct dm_io_client {
mempool_t pool;
struct bio_set bios;
};