diff options
Diffstat (limited to 'Documentation/video4linux/v4l2-framework.txt')
| -rw-r--r-- | Documentation/video4linux/v4l2-framework.txt | 1160 |
1 files changed, 0 insertions, 1160 deletions
diff --git a/Documentation/video4linux/v4l2-framework.txt b/Documentation/video4linux/v4l2-framework.txt deleted file mode 100644 index cbefc7902..000000000 --- a/Documentation/video4linux/v4l2-framework.txt +++ /dev/null @@ -1,1160 +0,0 @@ -Overview of the V4L2 driver framework -===================================== - -This text documents the various structures provided by the V4L2 framework and -their relationships. - - -Introduction ------------- - -The V4L2 drivers tend to be very complex due to the complexity of the -hardware: most devices have multiple ICs, export multiple device nodes in -/dev, and create also non-V4L2 devices such as DVB, ALSA, FB, I2C and input -(IR) devices. - -Especially the fact that V4L2 drivers have to setup supporting ICs to -do audio/video muxing/encoding/decoding makes it more complex than most. -Usually these ICs are connected to the main bridge driver through one or -more I2C busses, but other busses can also be used. Such devices are -called 'sub-devices'. - -For a long time the framework was limited to the video_device struct for -creating V4L device nodes and video_buf for handling the video buffers -(note that this document does not discuss the video_buf framework). - -This meant that all drivers had to do the setup of device instances and -connecting to sub-devices themselves. Some of this is quite complicated -to do right and many drivers never did do it correctly. - -There is also a lot of common code that could never be refactored due to -the lack of a framework. - -So this framework sets up the basic building blocks that all drivers -need and this same framework should make it much easier to refactor -common code into utility functions shared by all drivers. - -A good example to look at as a reference is the v4l2-pci-skeleton.c -source that is available in samples/v4l/. It is a skeleton driver for -a PCI capture card, and demonstrates how to use the V4L2 driver -framework. It can be used as a template for real PCI video capture driver. - -Structure of a driver ---------------------- - -All drivers have the following structure: - -1) A struct for each device instance containing the device state. - -2) A way of initializing and commanding sub-devices (if any). - -3) Creating V4L2 device nodes (/dev/videoX, /dev/vbiX and /dev/radioX) - and keeping track of device-node specific data. - -4) Filehandle-specific structs containing per-filehandle data; - -5) video buffer handling. - -This is a rough schematic of how it all relates: - - device instances - | - +-sub-device instances - | - \-V4L2 device nodes - | - \-filehandle instances - - -Structure of the framework --------------------------- - -The framework closely resembles the driver structure: it has a v4l2_device -struct for the device instance data, a v4l2_subdev struct to refer to -sub-device instances, the video_device struct stores V4L2 device node data -and the v4l2_fh struct keeps track of filehandle instances. - -The V4L2 framework also optionally integrates with the media framework. If a -driver sets the struct v4l2_device mdev field, sub-devices and video nodes -will automatically appear in the media framework as entities. - - -struct v4l2_device ------------------- - -Each device instance is represented by a struct v4l2_device (v4l2-device.h). -Very simple devices can just allocate this struct, but most of the time you -would embed this struct inside a larger struct. - -You must register the device instance: - - v4l2_device_register(struct device *dev, struct v4l2_device *v4l2_dev); - -Registration will initialize the v4l2_device struct. If the dev->driver_data -field is NULL, it will be linked to v4l2_dev. - -Drivers that want integration with the media device framework need to set -dev->driver_data manually to point to the driver-specific device structure -that embed the struct v4l2_device instance. This is achieved by a -dev_set_drvdata() call before registering the V4L2 device instance. They must -also set the struct v4l2_device mdev field to point to a properly initialized -and registered media_device instance. - -If v4l2_dev->name is empty then it will be set to a value derived from dev -(driver name followed by the bus_id, to be precise). If you set it up before -calling v4l2_device_register then it will be untouched. If dev is NULL, then -you *must* setup v4l2_dev->name before calling v4l2_device_register. - -You can use v4l2_device_set_name() to set the name based on a driver name and -a driver-global atomic_t instance. This will generate names like ivtv0, ivtv1, -etc. If the name ends with a digit, then it will insert a dash: cx18-0, -cx18-1, etc. This function returns the instance number. - -The first 'dev' argument is normally the struct device pointer of a pci_dev, -usb_interface or platform_device. It is rare for dev to be NULL, but it happens -with ISA devices or when one device creates multiple PCI devices, thus making -it impossible to associate v4l2_dev with a particular parent. - -You can also supply a notify() callback that can be called by sub-devices to -notify you of events. Whether you need to set this depends on the sub-device. -Any notifications a sub-device supports must be defined in a header in -include/media/<subdevice>.h. - -You unregister with: - - v4l2_device_unregister(struct v4l2_device *v4l2_dev); - -If the dev->driver_data field points to v4l2_dev, it will be reset to NULL. -Unregistering will also automatically unregister all subdevs from the device. - -If you have a hotpluggable device (e.g. a USB device), then when a disconnect -happens the parent device becomes invalid. Since v4l2_device has a pointer to -that parent device it has to be cleared as well to mark that the parent is -gone. To do this call: - - v4l2_device_disconnect(struct v4l2_device *v4l2_dev); - -This does *not* unregister the subdevs, so you still need to call the -v4l2_device_unregister() function for that. If your driver is not hotpluggable, -then there is no need to call v4l2_device_disconnect(). - -Sometimes you need to iterate over all devices registered by a specific -driver. This is usually the case if multiple device drivers use the same -hardware. E.g. the ivtvfb driver is a framebuffer driver that uses the ivtv -hardware. The same is true for alsa drivers for example. - -You can iterate over all registered devices as follows: - -static int callback(struct device *dev, void *p) -{ - struct v4l2_device *v4l2_dev = dev_get_drvdata(dev); - - /* test if this device was inited */ - if (v4l2_dev == NULL) - return 0; - ... - return 0; -} - -int iterate(void *p) -{ - struct device_driver *drv; - int err; - - /* Find driver 'ivtv' on the PCI bus. - pci_bus_type is a global. For USB busses use usb_bus_type. */ - drv = driver_find("ivtv", &pci_bus_type); - /* iterate over all ivtv device instances */ - err = driver_for_each_device(drv, NULL, p, callback); - put_driver(drv); - return err; -} - -Sometimes you need to keep a running counter of the device instance. This is -commonly used to map a device instance to an index of a module option array. - -The recommended approach is as follows: - -static atomic_t drv_instance = ATOMIC_INIT(0); - -static int drv_probe(struct pci_dev *pdev, const struct pci_device_id *pci_id) -{ - ... - state->instance = atomic_inc_return(&drv_instance) - 1; -} - -If you have multiple device nodes then it can be difficult to know when it is -safe to unregister v4l2_device for hotpluggable devices. For this purpose -v4l2_device has refcounting support. The refcount is increased whenever -video_register_device is called and it is decreased whenever that device node -is released. When the refcount reaches zero, then the v4l2_device release() -callback is called. You can do your final cleanup there. - -If other device nodes (e.g. ALSA) are created, then you can increase and -decrease the refcount manually as well by calling: - -void v4l2_device_get(struct v4l2_device *v4l2_dev); - -or: - -int v4l2_device_put(struct v4l2_device *v4l2_dev); - -Since the initial refcount is 1 you also need to call v4l2_device_put in the -disconnect() callback (for USB devices) or in the remove() callback (for e.g. -PCI devices), otherwise the refcount will never reach 0. - -struct v4l2_subdev ------------------- - -Many drivers need to communicate with sub-devices. These devices can do all -sort of tasks, but most commonly they handle audio and/or video muxing, -encoding or decoding. For webcams common sub-devices are sensors and camera -controllers. - -Usually these are I2C devices, but not necessarily. In order to provide the -driver with a consistent interface to these sub-devices the v4l2_subdev struct -(v4l2-subdev.h) was created. - -Each sub-device driver must have a v4l2_subdev struct. This struct can be -stand-alone for simple sub-devices or it might be embedded in a larger struct -if more state information needs to be stored. Usually there is a low-level -device struct (e.g. i2c_client) that contains the device data as setup -by the kernel. It is recommended to store that pointer in the private -data of v4l2_subdev using v4l2_set_subdevdata(). That makes it easy to go -from a v4l2_subdev to the actual low-level bus-specific device data. - -You also need a way to go from the low-level struct to v4l2_subdev. For the -common i2c_client struct the i2c_set_clientdata() call is used to store a -v4l2_subdev pointer, for other busses you may have to use other methods. - -Bridges might also need to store per-subdev private data, such as a pointer to -bridge-specific per-subdev private data. The v4l2_subdev structure provides -host private data for that purpose that can be accessed with -v4l2_get_subdev_hostdata() and v4l2_set_subdev_hostdata(). - -From the bridge driver perspective you load the sub-device module and somehow -obtain the v4l2_subdev pointer. For i2c devices this is easy: you call -i2c_get_clientdata(). For other busses something similar needs to be done. -Helper functions exists for sub-devices on an I2C bus that do most of this -tricky work for you. - -Each v4l2_subdev contains function pointers that sub-device drivers can -implement (or leave NULL if it is not applicable). Since sub-devices can do -so many different things and you do not want to end up with a huge ops struct -of which only a handful of ops are commonly implemented, the function pointers -are sorted according to category and each category has its own ops struct. - -The top-level ops struct contains pointers to the category ops structs, which -may be NULL if the subdev driver does not support anything from that category. - -It looks like this: - -struct v4l2_subdev_core_ops { - int (*log_status)(struct v4l2_subdev *sd); - int (*init)(struct v4l2_subdev *sd, u32 val); - ... -}; - -struct v4l2_subdev_tuner_ops { - ... -}; - -struct v4l2_subdev_audio_ops { - ... -}; - -struct v4l2_subdev_video_ops { - ... -}; - -struct v4l2_subdev_pad_ops { - ... -}; - -struct v4l2_subdev_ops { - const struct v4l2_subdev_core_ops *core; - const struct v4l2_subdev_tuner_ops *tuner; - const struct v4l2_subdev_audio_ops *audio; - const struct v4l2_subdev_video_ops *video; - const struct v4l2_subdev_pad_ops *video; -}; - -The core ops are common to all subdevs, the other categories are implemented -depending on the sub-device. E.g. a video device is unlikely to support the -audio ops and vice versa. - -This setup limits the number of function pointers while still making it easy -to add new ops and categories. - -A sub-device driver initializes the v4l2_subdev struct using: - - v4l2_subdev_init(sd, &ops); - -Afterwards you need to initialize subdev->name with a unique name and set the -module owner. This is done for you if you use the i2c helper functions. - -If integration with the media framework is needed, you must initialize the -media_entity struct embedded in the v4l2_subdev struct (entity field) by -calling media_entity_pads_init(), if the entity has pads: - - struct media_pad *pads = &my_sd->pads; - int err; - - err = media_entity_pads_init(&sd->entity, npads, pads); - -The pads array must have been previously initialized. There is no need to -manually set the struct media_entity function and name fields, but the -revision field must be initialized if needed. - -A reference to the entity will be automatically acquired/released when the -subdev device node (if any) is opened/closed. - -Don't forget to cleanup the media entity before the sub-device is destroyed: - - media_entity_cleanup(&sd->entity); - -If the subdev driver intends to process video and integrate with the media -framework, it must implement format related functionality using -v4l2_subdev_pad_ops instead of v4l2_subdev_video_ops. - -In that case, the subdev driver may set the link_validate field to provide -its own link validation function. The link validation function is called for -every link in the pipeline where both of the ends of the links are V4L2 -sub-devices. The driver is still responsible for validating the correctness -of the format configuration between sub-devices and video nodes. - -If link_validate op is not set, the default function -v4l2_subdev_link_validate_default() is used instead. This function ensures -that width, height and the media bus pixel code are equal on both source and -sink of the link. Subdev drivers are also free to use this function to -perform the checks mentioned above in addition to their own checks. - -There are currently two ways to register subdevices with the V4L2 core. The -first (traditional) possibility is to have subdevices registered by bridge -drivers. This can be done when the bridge driver has the complete information -about subdevices connected to it and knows exactly when to register them. This -is typically the case for internal subdevices, like video data processing units -within SoCs or complex PCI(e) boards, camera sensors in USB cameras or connected -to SoCs, which pass information about them to bridge drivers, usually in their -platform data. - -There are however also situations where subdevices have to be registered -asynchronously to bridge devices. An example of such a configuration is a Device -Tree based system where information about subdevices is made available to the -system independently from the bridge devices, e.g. when subdevices are defined -in DT as I2C device nodes. The API used in this second case is described further -below. - -Using one or the other registration method only affects the probing process, the -run-time bridge-subdevice interaction is in both cases the same. - -In the synchronous case a device (bridge) driver needs to register the -v4l2_subdev with the v4l2_device: - - int err = v4l2_device_register_subdev(v4l2_dev, sd); - -This can fail if the subdev module disappeared before it could be registered. -After this function was called successfully the subdev->dev field points to -the v4l2_device. - -If the v4l2_device parent device has a non-NULL mdev field, the sub-device -entity will be automatically registered with the media device. - -You can unregister a sub-device using: - - v4l2_device_unregister_subdev(sd); - -Afterwards the subdev module can be unloaded and sd->dev == NULL. - -You can call an ops function either directly: - - err = sd->ops->core->g_std(sd, &norm); - -but it is better and easier to use this macro: - - err = v4l2_subdev_call(sd, core, g_std, &norm); - -The macro will to the right NULL pointer checks and returns -ENODEV if subdev -is NULL, -ENOIOCTLCMD if either subdev->core or subdev->core->g_std is -NULL, or the actual result of the subdev->ops->core->g_std ops. - -It is also possible to call all or a subset of the sub-devices: - - v4l2_device_call_all(v4l2_dev, 0, core, g_std, &norm); - -Any subdev that does not support this ops is skipped and error results are -ignored. If you want to check for errors use this: - - err = v4l2_device_call_until_err(v4l2_dev, 0, core, g_std, &norm); - -Any error except -ENOIOCTLCMD will exit the loop with that error. If no -errors (except -ENOIOCTLCMD) occurred, then 0 is returned. - -The second argument to both calls is a group ID. If 0, then all subdevs are -called. If non-zero, then only those whose group ID match that value will -be called. Before a bridge driver registers a subdev it can set sd->grp_id -to whatever value it wants (it's 0 by default). This value is owned by the -bridge driver and the sub-device driver will never modify or use it. - -The group ID gives the bridge driver more control how callbacks are called. -For example, there may be multiple audio chips on a board, each capable of -changing the volume. But usually only one will actually be used when the -user want to change the volume. You can set the group ID for that subdev to -e.g. AUDIO_CONTROLLER and specify that as the group ID value when calling -v4l2_device_call_all(). That ensures that it will only go to the subdev -that needs it. - -If the sub-device needs to notify its v4l2_device parent of an event, then -it can call v4l2_subdev_notify(sd, notification, arg). This macro checks -whether there is a notify() callback defined and returns -ENODEV if not. -Otherwise the result of the notify() call is returned. - -The advantage of using v4l2_subdev is that it is a generic struct and does -not contain any knowledge about the underlying hardware. So a driver might -contain several subdevs that use an I2C bus, but also a subdev that is -controlled through GPIO pins. This distinction is only relevant when setting -up the device, but once the subdev is registered it is completely transparent. - - -In the asynchronous case subdevice probing can be invoked independently of the -bridge driver availability. The subdevice driver then has to verify whether all -the requirements for a successful probing are satisfied. This can include a -check for a master clock availability. If any of the conditions aren't satisfied -the driver might decide to return -EPROBE_DEFER to request further reprobing -attempts. Once all conditions are met the subdevice shall be registered using -the v4l2_async_register_subdev() function. Unregistration is performed using -the v4l2_async_unregister_subdev() call. Subdevices registered this way are -stored in a global list of subdevices, ready to be picked up by bridge drivers. - -Bridge drivers in turn have to register a notifier object with an array of -subdevice descriptors that the bridge device needs for its operation. This is -performed using the v4l2_async_notifier_register() call. To unregister the -notifier the driver has to call v4l2_async_notifier_unregister(). The former of -the two functions takes two arguments: a pointer to struct v4l2_device and a -pointer to struct v4l2_async_notifier. The latter contains a pointer to an array -of pointers to subdevice descriptors of type struct v4l2_async_subdev type. The -V4L2 core will then use these descriptors to match asynchronously registered -subdevices to them. If a match is detected the .bound() notifier callback is -called. After all subdevices have been located the .complete() callback is -called. When a subdevice is removed from the system the .unbind() method is -called. All three callbacks are optional. - - -V4L2 sub-device userspace API ------------------------------ - -Beside exposing a kernel API through the v4l2_subdev_ops structure, V4L2 -sub-devices can also be controlled directly by userspace applications. - -Device nodes named v4l-subdevX can be created in /dev to access sub-devices -directly. If a sub-device supports direct userspace configuration it must set -the V4L2_SUBDEV_FL_HAS_DEVNODE flag before being registered. - -After registering sub-devices, the v4l2_device driver can create device nodes -for all registered sub-devices marked with V4L2_SUBDEV_FL_HAS_DEVNODE by calling -v4l2_device_register_subdev_nodes(). Those device nodes will be automatically -removed when sub-devices are unregistered. - -The device node handles a subset of the V4L2 API. - -VIDIOC_QUERYCTRL -VIDIOC_QUERYMENU -VIDIOC_G_CTRL -VIDIOC_S_CTRL -VIDIOC_G_EXT_CTRLS -VIDIOC_S_EXT_CTRLS -VIDIOC_TRY_EXT_CTRLS - - The controls ioctls are identical to the ones defined in V4L2. They - behave identically, with the only exception that they deal only with - controls implemented in the sub-device. Depending on the driver, those - controls can be also be accessed through one (or several) V4L2 device - nodes. - -VIDIOC_DQEVENT -VIDIOC_SUBSCRIBE_EVENT -VIDIOC_UNSUBSCRIBE_EVENT - - The events ioctls are identical to the ones defined in V4L2. They - behave identically, with the only exception that they deal only with - events generated by the sub-device. Depending on the driver, those - events can also be reported by one (or several) V4L2 device nodes. - - Sub-device drivers that want to use events need to set the - V4L2_SUBDEV_USES_EVENTS v4l2_subdev::flags and initialize - v4l2_subdev::nevents to events queue depth before registering the - sub-device. After registration events can be queued as usual on the - v4l2_subdev::devnode device node. - - To properly support events, the poll() file operation is also - implemented. - -Private ioctls - - All ioctls not in the above list are passed directly to the sub-device - driver through the core::ioctl operation. - - -I2C sub-device drivers ----------------------- - -Since these drivers are so common, special helper functions are available to -ease the use of these drivers (v4l2-common.h). - -The recommended method of adding v4l2_subdev support to an I2C driver is to -embed the v4l2_subdev struct into the state struct that is created for each -I2C device instance. Very simple devices have no state struct and in that case -you can just create a v4l2_subdev directly. - -A typical state struct would look like this (where 'chipname' is replaced by -the name of the chip): - -struct chipname_state { - struct v4l2_subdev sd; - ... /* additional state fields */ -}; - -Initialize the v4l2_subdev struct as follows: - - v4l2_i2c_subdev_init(&state->sd, client, subdev_ops); - -This function will fill in all the fields of v4l2_subdev and ensure that the -v4l2_subdev and i2c_client both point to one another. - -You should also add a helper inline function to go from a v4l2_subdev pointer -to a chipname_state struct: - -static inline struct chipname_state *to_state(struct v4l2_subdev *sd) -{ - return container_of(sd, struct chipname_state, sd); -} - -Use this to go from the v4l2_subdev struct to the i2c_client struct: - - struct i2c_client *client = v4l2_get_subdevdata(sd); - -And this to go from an i2c_client to a v4l2_subdev struct: - - struct v4l2_subdev *sd = i2c_get_clientdata(client); - -Make sure to call v4l2_device_unregister_subdev(sd) when the remove() callback -is called. This will unregister the sub-device from the bridge driver. It is -safe to call this even if the sub-device was never registered. - -You need to do this because when the bridge driver destroys the i2c adapter -the remove() callbacks are called of the i2c devices on that adapter. -After that the corresponding v4l2_subdev structures are invalid, so they -have to be unregistered first. Calling v4l2_device_unregister_subdev(sd) -from the remove() callback ensures that this is always done correctly. - - -The bridge driver also has some helper functions it can use: - -struct v4l2_subdev *sd = v4l2_i2c_new_subdev(v4l2_dev, adapter, - "module_foo", "chipid", 0x36, NULL); - -This loads the given module (can be NULL if no module needs to be loaded) and -calls i2c_new_device() with the given i2c_adapter and chip/address arguments. -If all goes well, then it registers the subdev with the v4l2_device. - -You can also use the last argument of v4l2_i2c_new_subdev() to pass an array -of possible I2C addresses that it should probe. These probe addresses are -only used if the previous argument is 0. A non-zero argument means that you -know the exact i2c address so in that case no probing will take place. - -Both functions return NULL if something went wrong. - -Note that the chipid you pass to v4l2_i2c_new_subdev() is usually -the same as the module name. It allows you to specify a chip variant, e.g. -"saa7114" or "saa7115". In general though the i2c driver autodetects this. -The use of chipid is something that needs to be looked at more closely at a -later date. It differs between i2c drivers and as such can be confusing. -To see which chip variants are supported you can look in the i2c driver code -for the i2c_device_id table. This lists all the possibilities. - -There are two more helper functions: - -v4l2_i2c_new_subdev_cfg: this function adds new irq and platform_data -arguments and has both 'addr' and 'probed_addrs' arguments: if addr is not -0 then that will be used (non-probing variant), otherwise the probed_addrs -are probed. - -For example: this will probe for address 0x10: - -struct v4l2_subdev *sd = v4l2_i2c_new_subdev_cfg(v4l2_dev, adapter, - "module_foo", "chipid", 0, NULL, 0, I2C_ADDRS(0x10)); - -v4l2_i2c_new_subdev_board uses an i2c_board_info struct which is passed -to the i2c driver and replaces the irq, platform_data and addr arguments. - -If the subdev supports the s_config core ops, then that op is called with -the irq and platform_data arguments after the subdev was setup. The older -v4l2_i2c_new_(probed_)subdev functions will call s_config as well, but with -irq set to 0 and platform_data set to NULL. - -struct video_device -------------------- - -The actual device nodes in the /dev directory are created using the -video_device struct (v4l2-dev.h). This struct can either be allocated -dynamically or embedded in a larger struct. - -To allocate it dynamically use: - - struct video_device *vdev = video_device_alloc(); - - if (vdev == NULL) - return -ENOMEM; - - vdev->release = video_device_release; - -If you embed it in a larger struct, then you must set the release() -callback to your own function: - - struct video_device *vdev = &my_vdev->vdev; - - vdev->release = my_vdev_release; - -The release callback must be set and it is called when the last user -of the video device exits. - -The default video_device_release() callback just calls kfree to free the -allocated memory. - -There is also a video_device_release_empty() function that does nothing -(is empty) and can be used if the struct is embedded and there is nothing -to do when it is released. - -You should also set these fields: - -- v4l2_dev: must be set to the v4l2_device parent device. - -- name: set to something descriptive and unique. - -- vfl_dir: set this to VFL_DIR_RX for capture devices (VFL_DIR_RX has value 0, - so this is normally already the default), set to VFL_DIR_TX for output - devices and VFL_DIR_M2M for mem2mem (codec) devices. - -- fops: set to the v4l2_file_operations struct. - -- ioctl_ops: if you use the v4l2_ioctl_ops to simplify ioctl maintenance - (highly recommended to use this and it might become compulsory in the - future!), then set this to your v4l2_ioctl_ops struct. The vfl_type and - vfl_dir fields are used to disable ops that do not match the type/dir - combination. E.g. VBI ops are disabled for non-VBI nodes, and output ops - are disabled for a capture device. This makes it possible to provide - just one v4l2_ioctl_ops struct for both vbi and video nodes. - -- lock: leave to NULL if you want to do all the locking in the driver. - Otherwise you give it a pointer to a struct mutex_lock and before the - unlocked_ioctl file operation is called this lock will be taken by the - core and released afterwards. See the next section for more details. - -- queue: a pointer to the struct vb2_queue associated with this device node. - If queue is non-NULL, and queue->lock is non-NULL, then queue->lock is - used for the queuing ioctls (VIDIOC_REQBUFS, CREATE_BUFS, QBUF, DQBUF, - QUERYBUF, PREPARE_BUF, STREAMON and STREAMOFF) instead of the lock above. - That way the vb2 queuing framework does not have to wait for other ioctls. - This queue pointer is also used by the vb2 helper functions to check for - queuing ownership (i.e. is the filehandle calling it allowed to do the - operation). - -- prio: keeps track of the priorities. Used to implement VIDIOC_G/S_PRIORITY. - If left to NULL, then it will use the struct v4l2_prio_state in v4l2_device. - If you want to have a separate priority state per (group of) device node(s), - then you can point it to your own struct v4l2_prio_state. - -- dev_parent: you only set this if v4l2_device was registered with NULL as - the parent device struct. This only happens in cases where one hardware - device has multiple PCI devices that all share the same v4l2_device core. - - The cx88 driver is an example of this: one core v4l2_device struct, but - it is used by both a raw video PCI device (cx8800) and a MPEG PCI device - (cx8802). Since the v4l2_device cannot be associated with two PCI devices - at the same time it is setup without a parent device. But when the struct - video_device is initialized you *do* know which parent PCI device to use and - so you set dev_device to the correct PCI device. - -If you use v4l2_ioctl_ops, then you should set .unlocked_ioctl to video_ioctl2 -in your v4l2_file_operations struct. - -Do not use .ioctl! This is deprecated and will go away in the future. - -In some cases you want to tell the core that a function you had specified in -your v4l2_ioctl_ops should be ignored. You can mark such ioctls by calling this -function before video_device_register is called: - -void v4l2_disable_ioctl(struct video_device *vdev, unsigned int cmd); - -This tends to be needed if based on external factors (e.g. which card is -being used) you want to turns off certain features in v4l2_ioctl_ops without -having to make a new struct. - -The v4l2_file_operations struct is a subset of file_operations. The main -difference is that the inode argument is omitted since it is never used. - -If integration with the media framework is needed, you must initialize the -media_entity struct embedded in the video_device struct (entity field) by -calling media_entity_pads_init(): - - struct media_pad *pad = &my_vdev->pad; - int err; - - err = media_entity_pads_init(&vdev->entity, 1, pad); - -The pads array must have been previously initialized. There is no need to -manually set the struct media_entity type and name fields. - -A reference to the entity will be automatically acquired/released when the -video device is opened/closed. - -ioctls and locking ------------------- - -The V4L core provides optional locking services. The main service is the -lock field in struct video_device, which is a pointer to a mutex. If you set -this pointer, then that will be used by unlocked_ioctl to serialize all ioctls. - -If you are using the videobuf2 framework, then there is a second lock that you -can set: video_device->queue->lock. If set, then this lock will be used instead -of video_device->lock to serialize all queuing ioctls (see the previous section -for the full list of those ioctls). - -The advantage of using a different lock for the queuing ioctls is that for some -drivers (particularly USB drivers) certain commands such as setting controls -can take a long time, so you want to use a separate lock for the buffer queuing -ioctls. That way your VIDIOC_DQBUF doesn't stall because the driver is busy -changing the e.g. exposure of the webcam. - -Of course, you can always do all the locking yourself by leaving both lock -pointers at NULL. - -If you use the old videobuf then you must pass the video_device lock to the -videobuf queue initialize function: if videobuf has to wait for a frame to -arrive, then it will temporarily unlock the lock and relock it afterwards. If -your driver also waits in the code, then you should do the same to allow other -processes to access the device node while the first process is waiting for -something. - -In the case of videobuf2 you will need to implement the wait_prepare and -wait_finish callbacks to unlock/lock if applicable. If you use the queue->lock -pointer, then you can use the helper functions vb2_ops_wait_prepare/finish. - -The implementation of a hotplug disconnect should also take the lock from -video_device before calling v4l2_device_disconnect. If you are also using -video_device->queue->lock, then you have to first lock video_device->queue->lock -followed by video_device->lock. That way you can be sure no ioctl is running -when you call v4l2_device_disconnect. - -video_device registration -------------------------- - -Next you register the video device: this will create the character device -for you. - - err = video_register_device(vdev, VFL_TYPE_GRABBER, -1); - if (err) { - video_device_release(vdev); /* or kfree(my_vdev); */ - return err; - } - -If the v4l2_device parent device has a non-NULL mdev field, the video device -entity will be automatically registered with the media device. - -Which device is registered depends on the type argument. The following -types exist: - -VFL_TYPE_GRABBER: videoX for video input/output devices -VFL_TYPE_VBI: vbiX for vertical blank data (i.e. closed captions, teletext) -VFL_TYPE_RADIO: radioX for radio tuners -VFL_TYPE_SDR: swradioX for Software Defined Radio tuners - -The last argument gives you a certain amount of control over the device -device node number used (i.e. the X in videoX). Normally you will pass -1 -to let the v4l2 framework pick the first free number. But sometimes users -want to select a specific node number. It is common that drivers allow -the user to select a specific device node number through a driver module -option. That number is then passed to this function and video_register_device -will attempt to select that device node number. If that number was already -in use, then the next free device node number will be selected and it -will send a warning to the kernel log. - -Another use-case is if a driver creates many devices. In that case it can -be useful to place different video devices in separate ranges. For example, -video capture devices start at 0, video output devices start at 16. -So you can use the last argument to specify a minimum device node number -and the v4l2 framework will try to pick the first free number that is equal -or higher to what you passed. If that fails, then it will just pick the -first free number. - -Since in this case you do not care about a warning about not being able -to select the specified device node number, you can call the function -video_register_device_no_warn() instead. - -Whenever a device node is created some attributes are also created for you. -If you look in /sys/class/video4linux you see the devices. Go into e.g. -video0 and you will see 'name', 'dev_debug' and 'index' attributes. The 'name' -attribute is the 'name' field of the video_device struct. The 'dev_debug' attribute -can be used to enable core debugging. See the next section for more detailed -information on this. - -The 'index' attribute is the index of the device node: for each call to -video_register_device() the index is just increased by 1. The first video -device node you register always starts with index 0. - -Users can setup udev rules that utilize the index attribute to make fancy -device names (e.g. 'mpegX' for MPEG video capture device nodes). - -After the device was successfully registered, then you can use these fields: - -- vfl_type: the device type passed to video_register_device. -- minor: the assigned device minor number. -- num: the device node number (i.e. the X in videoX). -- index: the device index number. - -If the registration failed, then you need to call video_device_release() -to free the allocated video_device struct, or free your own struct if the -video_device was embedded in it. The vdev->release() callback will never -be called if the registration failed, nor should you ever attempt to -unregister the device if the registration failed. - -video device debugging ----------------------- - -The 'dev_debug' attribute that is created for each video, vbi, radio or swradio -device in /sys/class/video4linux/<devX>/ allows you to enable logging of -file operations. - -It is a bitmask and the following bits can be set: - -0x01: Log the ioctl name and error code. VIDIOC_(D)QBUF ioctls are only logged - if bit 0x08 is also set. -0x02: Log the ioctl name arguments and error code. VIDIOC_(D)QBUF ioctls are - only logged if bit 0x08 is also set. -0x04: Log the file operations open, release, read, write, mmap and - get_unmapped_area. The read and write operations are only logged if - bit 0x08 is also set. -0x08: Log the read and write file operations and the VIDIOC_QBUF and - VIDIOC_DQBUF ioctls. -0x10: Log the poll file operation. - -video_device cleanup --------------------- - -When the video device nodes have to be removed, either during the unload -of the driver or because the USB device was disconnected, then you should -unregister them: - - video_unregister_device(vdev); - -This will remove the device nodes from sysfs (causing udev to remove them -from /dev). - -After video_unregister_device() returns no new opens can be done. However, -in the case of USB devices some application might still have one of these -device nodes open. So after the unregister all file operations (except -release, of course) will return an error as well. - -When the last user of the video device node exits, then the vdev->release() -callback is called and you can do the final cleanup there. - -Don't forget to cleanup the media entity associated with the video device if -it has been initialized: - - media_entity_cleanup(&vdev->entity); - -This can be done from the release callback. - - -video_device helper functions ------------------------------ - -There are a few useful helper functions: - -- file/video_device private data - -You can set/get driver private data in the video_device struct using: - -void *video_get_drvdata(struct video_device *vdev); -void video_set_drvdata(struct video_device *vdev, void *data); - -Note that you can safely call video_set_drvdata() before calling -video_register_device(). - -And this function: - -struct video_device *video_devdata(struct file *file); - -returns the video_device belonging to the file struct. - -The video_drvdata function combines video_get_drvdata with video_devdata: - -void *video_drvdata(struct file *file); - -You can go from a video_device struct to the v4l2_device struct using: - -struct v4l2_device *v4l2_dev = vdev->v4l2_dev; - -- Device node name - -The video_device node kernel name can be retrieved using - -const char *video_device_node_name(struct video_device *vdev); - -The name is used as a hint by userspace tools such as udev. The function -should be used where possible instead of accessing the video_device::num and -video_device::minor fields. - - -video buffer helper functions ------------------------------ - -The v4l2 core API provides a set of standard methods (called "videobuf") -for dealing with video buffers. Those methods allow a driver to implement -read(), mmap() and overlay() in a consistent way. There are currently -methods for using video buffers on devices that supports DMA with -scatter/gather method (videobuf-dma-sg), DMA with linear access -(videobuf-dma-contig), and vmalloced buffers, mostly used on USB drivers -(videobuf-vmalloc). - -Please see Documentation/video4linux/videobuf for more information on how -to use the videobuf layer. - -struct v4l2_fh --------------- - -struct v4l2_fh provides a way to easily keep file handle specific data -that is used by the V4L2 framework. New drivers must use struct v4l2_fh -since it is also used to implement priority handling (VIDIOC_G/S_PRIORITY). - -The users of v4l2_fh (in the V4L2 framework, not the driver) know -whether a driver uses v4l2_fh as its file->private_data pointer by -testing the V4L2_FL_USES_V4L2_FH bit in video_device->flags. This bit is -set whenever v4l2_fh_init() is called. - -struct v4l2_fh is allocated as a part of the driver's own file handle -structure and file->private_data is set to it in the driver's open -function by the driver. - -In many cases the struct v4l2_fh will be embedded in a larger structure. -In that case you should call v4l2_fh_init+v4l2_fh_add in open() and -v4l2_fh_del+v4l2_fh_exit in release(). - -Drivers can extract their own file handle structure by using the container_of -macro. Example: - -struct my_fh { - int blah; - struct v4l2_fh fh; -}; - -... - -int my_open(struct file *file) -{ - struct my_fh *my_fh; - struct video_device *vfd; - int ret; - - ... - - my_fh = kzalloc(sizeof(*my_fh), GFP_KERNEL); - - ... - - v4l2_fh_init(&my_fh->fh, vfd); - - ... - - file->private_data = &my_fh->fh; - v4l2_fh_add(&my_fh->fh); - return 0; -} - -int my_release(struct file *file) -{ - struct v4l2_fh *fh = file->private_data; - struct my_fh *my_fh = container_of(fh, struct my_fh, fh); - - ... - v4l2_fh_del(&my_fh->fh); - v4l2_fh_exit(&my_fh->fh); - kfree(my_fh); - return 0; -} - -Below is a short description of the v4l2_fh functions used: - -void v4l2_fh_init(struct v4l2_fh *fh, struct video_device *vdev) - - Initialise the file handle. This *MUST* be performed in the driver's - v4l2_file_operations->open() handler. - -void v4l2_fh_add(struct v4l2_fh *fh) - - Add a v4l2_fh to video_device file handle list. Must be called once the - file handle is completely initialized. - -void v4l2_fh_del(struct v4l2_fh *fh) - - Unassociate the file handle from video_device(). The file handle - exit function may now be called. - -void v4l2_fh_exit(struct v4l2_fh *fh) - - Uninitialise the file handle. After uninitialisation the v4l2_fh - memory can be freed. - - -If struct v4l2_fh is not embedded, then you can use these helper functions: - -int v4l2_fh_open(struct file *filp) - - This allocates a struct v4l2_fh, initializes it and adds it to the struct - video_device associated with the file struct. - -int v4l2_fh_release(struct file *filp) - - This deletes it from the struct video_device associated with the file - struct, uninitialised the v4l2_fh and frees it. - -These two functions can be plugged into the v4l2_file_operation's open() and -release() ops. - - -Several drivers need to do something when the first file handle is opened and -when the last file handle closes. Two helper functions were added to check -whether the v4l2_fh struct is the only open filehandle of the associated -device node: - -int v4l2_fh_is_singular(struct v4l2_fh *fh) - - Returns 1 if the file handle is the only open file handle, else 0. - -int v4l2_fh_is_singular_file(struct file *filp) - - Same, but it calls v4l2_fh_is_singular with filp->private_data. - - -V4L2 events ------------ - -The V4L2 events provide a generic way to pass events to user space. -The driver must use v4l2_fh to be able to support V4L2 events. - -Events are defined by a type and an optional ID. The ID may refer to a V4L2 -object such as a control ID. If unused, then the ID is 0. - -When the user subscribes to an event the driver will allocate a number of -kevent structs for that event. So every (type, ID) event tuple will have -its own set of kevent structs. This guarantees that if a driver is generating -lots of events of one type in a short time, then that will not overwrite -events of another type. - -But if you get more events of one type than the number of kevents that were -reserved, then the oldest event will be dropped and the new one added. - -Furthermore, the internal struct v4l2_subscribed_event has merge() and -replace() callbacks which drivers can set. These callbacks are called when -a new event is raised and there is no more room. The replace() callback -allows you to replace the payload of the old event with that of the new event, -merging any relevant data from the old payload into the new payload that -replaces it. It is called when this event type has only one kevent struct -allocated. The merge() callback allows you to merge the oldest event payload -into that of the second-oldest event payload. It is called when there are two -or more kevent structs allocated. - -This way no status information is lost, just the intermediate steps leading -up to that state. - -A good example of these replace/merge callbacks is in v4l2-event.c: -ctrls_replace() and ctrls_merge() callbacks for the control event. - -Note: these callbacks can be called from interrupt context, so they must be -fast. - -Useful functions: - -void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev) - - Queue events to video device. The driver's only responsibility is to fill - in the type and the data fields. The other fields will be filled in by - V4L2. - -int v4l2_event_subscribe(struct v4l2_fh *fh, - struct v4l2_event_subscription *sub, unsigned elems, - const struct v4l2_subscribed_event_ops *ops) - - The video_device->ioctl_ops->vidioc_subscribe_event must check the driver - is able to produce events with specified event id. Then it calls - v4l2_event_subscribe() to subscribe the event. - - The elems argument is the size of the event queue for this event. If it is 0, - then the framework will fill in a default value (this depends on the event - type). - - The ops argument allows the driver to specify a number of callbacks: - * add: called when a new listener gets added (subscribing to the same - event twice will only cause this callback to get called once) - * del: called when a listener stops listening - * replace: replace event 'old' with event 'new'. - * merge: merge event 'old' into event 'new'. - All 4 callbacks are optional, if you don't want to specify any callbacks - the ops argument itself maybe NULL. - -int v4l2_event_unsubscribe(struct v4l2_fh *fh, - struct v4l2_event_subscription *sub) - - vidioc_unsubscribe_event in struct v4l2_ioctl_ops. A driver may use - v4l2_event_unsubscribe() directly unless it wants to be involved in - unsubscription process. - - The special type V4L2_EVENT_ALL may be used to unsubscribe all events. The - drivers may want to handle this in a special way. - -int v4l2_event_pending(struct v4l2_fh *fh) - - Returns the number of pending events. Useful when implementing poll. - -Events are delivered to user space through the poll system call. The driver -can use v4l2_fh->wait (a wait_queue_head_t) as the argument for poll_wait(). - -There are standard and private events. New standard events must use the -smallest available event type. The drivers must allocate their events from -their own class starting from class base. Class base is -V4L2_EVENT_PRIVATE_START + n * 1000 where n is the lowest available number. -The first event type in the class is reserved for future use, so the first -available event type is 'class base + 1'. - -An example on how the V4L2 events may be used can be found in the OMAP -3 ISP driver (drivers/media/platform/omap3isp). - -A subdev can directly send an event to the v4l2_device notify function with -V4L2_DEVICE_NOTIFY_EVENT. This allows the bridge to map the subdev that sends -the event to the video node(s) associated with the subdev that need to be -informed about such an event. - -V4L2 clocks ------------ - -Many subdevices, like camera sensors, TV decoders and encoders, need a clock -signal to be supplied by the system. Often this clock is supplied by the -respective bridge device. The Linux kernel provides a Common Clock Framework for -this purpose. However, it is not (yet) available on all architectures. Besides, -the nature of the multi-functional (clock, data + synchronisation, I2C control) -connection of subdevices to the system might impose special requirements on the -clock API usage. E.g. V4L2 has to support clock provider driver unregistration -while a subdevice driver is holding a reference to the clock. For these reasons -a V4L2 clock helper API has been developed and is provided to bridge and -subdevice drivers. - -The API consists of two parts: two functions to register and unregister a V4L2 -clock source: v4l2_clk_register() and v4l2_clk_unregister() and calls to control -a clock object, similar to the respective generic clock API calls: -v4l2_clk_get(), v4l2_clk_put(), v4l2_clk_enable(), v4l2_clk_disable(), -v4l2_clk_get_rate(), and v4l2_clk_set_rate(). Clock suppliers have to provide -clock operations that will be called when clock users invoke respective API -methods. - -It is expected that once the CCF becomes available on all relevant -architectures this API will be removed. |