According to Wikipedia, a plug-in is defined as: “In computing, a plug-in is a software component that adds a specific feature to an existing computer program.” Therefore, a plugin should be flexibly configurable and easily load the content specified in the configuration.
Thanks to Python’s inherent dynamic nature, implementing plugin systems becomes even more flexible. Existing dynamic plugins typically rely on Python’s namespaces and dynamic import capabilities to discover and import external dependencies.
For specific principles, refer to Creating and discovering plugins: https://docs.pytest.org/en/latest/index.html
Plugin Frameworks
pluggy
pluggy evolved from pytest. It provides peripheral plugin support for pytest. When developers need to extend pytest‘s functionality, they can create corresponding plugins based on pytest‘s specifications. After installing these plugins into the environment, pytest can automatically recognize them.
The underlying principle involves creating a hookspec = pluggy.HookspecMarker("eggsample") to mark the plugin specification, and then using hookimpl = pluggy.HookimplMarker("eggsample") to mark the plugin’s implementation.
Specification:
python
import pluggy
hookspec = pluggy.HookspecMarker("eggsample")
@hookspec
def eggsample_add_ingredients(ingredients: tuple):
"""Have a look at the ingredients and offer your own.
:param ingredients: the ingredients, don't touch them!
:return: a list of ingredients
"""
@hookspec
def eggsample_prep_condiments(condiments: dict):
"""Reorganize the condiments tray to your heart's content.
:param condiments: some sauces and stuff
:return: a witty comment about your activity
"""
Implementation:
python
import pluggy
hookimpl = pluggy.HookimplMarker("eggsample")
"""Marker to be imported and used in plugins (and for own implementations)"""
class ExamplePluggy:
@hookimpl
def eggsample_add_ingredients(self):
spices = ["salt", "pepper"]
you_can_never_have_enough_eggs = ["egg", "egg"]
ingredients = spices + you_can_never_have_enough_eggs
return ingredients
@hookimpl
def eggsample_prep_condiments(self, condiments):
condiments["mint sauce"] = 1
Then, load the plugin specifications and implementations into the plugin manager class. To discover plugins developed by others, call the load_setuptools_entrypoints method to look up other plugins already available under the specified namespace.
python
import itertools
import random
import pluggy
def get_plugin_manager():
pm = pluggy.PluginManager("eggsample")
pm.add_hookspecs(hookspecs)
pm.load_setuptools_entrypoints("eggsample")
pm.register(ExamplePluggy)
return pm
During usage, call pm.hook.eggsample_add_ingredients and pass the parameters.
External plugins simply need to implement according to the plugin specification:
python
import eggsample
@eggsample.hookimpl
def eggsample_add_ingredients(ingredients):
"""Here the caller expects us to return a list."""
if "egg" in ingredients:
spam = ["lovely spam", "wonderous spam"]
else:
spam = ["splendiferous spam", "magnificent spam"]
return spam
@eggsample.hookimpl
def eggsample_prep_condiments(condiments):
"""Here the caller passes a mutable object, so we mess with it directly."""
try:
del condiments["steak sauce"]
except KeyError:
pass
condiments["spam sauce"] = 42
return "Now this is what I call a condiments tray!"
And specify the same namespace in the packaging information (setup.py):
python
from setuptools import setup
setup(
name="eggsample-spam",
install_requires="eggsample",
entry_points={"eggsample": ["spam = eggsample_spam"]},
py_modules=["eggsample_spam"],
)
The underlying principle also uses Python’s from importlib.metadata import entry_points to find packages registered in the Python interpreter’s entry_points and retrieve the required content based on the namespace.
stevedore
stevedore is a plugin tool developed and maintained by Openstack. This plugin provides plugin functionality for Openstack’s ceilometer.
stevedore recommends using inheritance to standardize plugin interfaces.
First, create a base plugin class:
python
# stevedore/example/base.py
# -*- coding: utf-8 -*-
# Copyright (C) 2020 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import abc
class FormatterBase(metaclass=abc.ABCMeta):
"""Base class for example plugin used in the tutorial.
"""
def __init__(self, max_width=60):
self.max_width = max_width
@abc.abstractmethod
def format(self, data):
"""Format the data and return unicode text.
:param data: A dictionary with string keys and simple types as
values.
:type data: dict(str:?)
:returns: Iterable producing the formatted text.
"""
Then implement a simple plugin:
python
# stevedore/example/simple.py
# Copyright (C) 2020 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from stevedore.example import base
class Simple(base.FormatterBase):
"""A very basic formatter."""
def format(self, data):
"""Format the data and return unicode text.
:param data: A dictionary with string keys and simple types as
values.
:type data: dict(str:?)
"""
for name, value in sorted(data.items()):
line = '{name} = {value}\n'.format(
name=name,
value=value,
)
yield line
Finally, package it. During packaging, register the plugin in entry_points.
python
# stevedore/example/setup.py
# Copyright (C) 2020 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from setuptools import find_packages
from setuptools import setup
setup(
name='stevedore-examples',
version='1.0',
description='Demonstration package for stevedore',
author='Doug Hellmann',
author_email='doug@doughellmann.com',
url='http://opendev.org/openstack/stevedore',
classifiers=['Development Status :: 3 - Alpha',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.5',
'Intended Audience :: Developers',
'Environment :: Console',
],
platforms=['Any'],
scripts=[],
provides=['stevedore.examples',
],
packages=find_packages(),
include_package_data=True,
entry_points={
'stevedore.example.formatter': [
'simple = stevedore.example.simple:Simple',
'plain = stevedore.example.simple:Simple',
],
},
zip_safe=False,
)
Create another plugin project and implement the plugin:
python
# stevedore/example2/fields.py
# -*- coding: utf-8 -*-
# Copyright (C) 2020 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import textwrap
from stevedore.example import base
class FieldList(base.FormatterBase):
"""Format values as a reStructuredText field list.
For example::
: name1 : value
: name2 : value
: name3 : a long value
will be wrapped with
a hanging indent
"""
def format(self, data):
"""Format the data and return unicode text.
:param data: A dictionary with string keys and simple types as
values.
:type data: dict(str:?)
"""
for name, value in sorted(data.items()):
full_text = ': {name} : {value}'.format(
name=name,
value=value,
)
wrapped_text = textwrap.fill(
full_text,
initial_indent='',
subsequent_indent=' ',
width=self.max_width,
)
yield wrapped_text + '\n'
Similarly, package it and configure the registration information, registering the plugin under the same namespace:
python
# stevedore/example2/setup.py
# Copyright (C) 2020 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from setuptools import find_packages
from setuptools import setup
setup(
name='stevedore-examples2',
version='1.0',
description='Demonstration package for stevedore',
author='Doug Hellmann',
author_email='doug@doughellmann.com',
url='http://opendev.org/openstack/stevedore',
classifiers=['Development Status :: 3 - Alpha',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.5',
'Intended Audience :: Developers',
'Environment :: Console',
],
platforms=['Any'],
scripts=[],
provides=['stevedore.examples2',
],
packages=find_packages(),
include_package_data=True,
entry_points={
'stevedore.example.formatter': [
'field = stevedore.example2.fields:FieldList',
],
},
zip_safe=False,
)
During usage, you can call it via the driver method:
python
# stevedore/example/load_as_driver.py
# Copyright (C) 2020 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
from stevedore import driver
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument(
'format',
nargs='?',
default='simple',
help='the output format',
)
parser.add_argument(
'--width',
default=60,
type=int,
help='maximum output width for text',
)
parsed_args = parser.parse_args()
data = {
'a': 'A',
'b': 'B',
'long': 'word ' * 80,
}
mgr = driver.DriverManager(
namespace='stevedore.example.formatter',
name=parsed_args.format,
invoke_on_load=True,
invoke_args=(parsed_args.width,),
)
for chunk in mgr.driver.format(data):
print(chunk, end='')
Or call it via the extensions method:
python
# stevedore/example/load_as_extension.py
# Copyright (C) 2020 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
from stevedore import extension
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument(
'--width',
default=60,
type=int,
help='maximum output width for text',
)
parsed_args = parser.parse_args()
data = {
'a': 'A',
'b': 'B',
'long': 'word ' * 80,
}
mgr = extension.ExtensionManager(
namespace='stevedore.example.formatter',
invoke_on_load=True,
invoke_args=(parsed_args.width,),
)
def format_data(ext, data):
return (ext.name, ext.obj.format(data))
results = mgr.map(format_data, data)
for name, result in results:
print('Formatter: {0}'.format(name))
for chunk in result:
print(chunk, end='')
print('')
Practice
Based on practical development experience, using stevedore is recommended. Summarized advantages are as follows:
- Defines interface specifications by designing base classes.
- Plugin invocation is more flexible.
stevedoreoffers greater sophistication in terms of usage complexity.