Frequently Asked Questions (FAQ)

The following is a list of frequently asked questions related to the Dragonfly speech recognition framework.

General Questions

What is Dragonfly?

Dragonfly is a speech recognition framework for Python that makes it convenient to create custom commands to use with speech recognition software. It was written to make it very easy for Python macros, scripts, and applications to interface with speech recognition engines. Its design allows speech commands and grammar objects to be treated as first-class Python objects.

Dragonfly can be used for general programming by voice. It is flexible enough to allow programming in any language, not just Python. It can also be used for speech-enabling applications, automating computer activities and dictating prose.

Which speech recognition software and operating systems are supported?

Dragonfly supports the following speech recognition (SR) engines:

  • Dragon, a product of Nuance. All versions up to 16 (the latest) are supported. Home, Professional Individual and previous similar editions of Dragon are supported. Other editions may work too.

  • Windows Speech Recognition (WSR), included with Microsoft Windows Vista, Windows 7+, and freely available for Windows XP.

  • Kaldi

  • CMU Pocket Sphinx

Dragonfly has cross platform support for Windows, macOS and Linux (X11-only). The following table shows which engines are available on which platforms:

Operating system

Available SR engines

Windows

DNS, WSR, Kaldi, Sphinx

Linux

Kaldi, Sphinx

macOS

Kaldi, Sphinx

Please note that editions of Dragon for macOS are not supported.

Windows-only speech recognition software (DNS and WSR) may be used to control Linux or macOS machines via Aenea, a client-server library for using Dragonfly voice macros on remote hosts.

Dragonfly’s X11 support is dependent on the external xdotool, wmctrl and xsel programs. These programs are available for most Linux distributions. They are also available for other operating systems, such as FreeBSD. At present, since xdotool is not available for it, Dragonfly will not work properly in a Cygwin environment.

Where can I find examples Dragonfly command modules?

There is a list of repositories and other projects containing Dragonfly command modules under the Related resources -> Command modules section of the documentation. There are also example command modules in dragonfly/examples.

What is the difference between dragonfly and dragonfly2?

Dragonfly is the original project written by Christo Butcher. It is no longer maintained. Dragonfly2 is a fork of Dragonfly that uses a different distribution name.

It is important to note that the import name is still “dragonfly”:

from dragonfly import Grammar, MappingRule, Key, Text, Mouse, Dictation

Dragonfly2 is intended to be backwards-compatible continuation of the original project. Many problems are fixed in this version. It supports alternative speech recognition engine backends (e.g. the Kaldi engine), works with Python 3 and has cross-support for macOS and the X Window System (X11). Dragonfly2 also has some new features not found in the old version.

See the changelog for the full list of changes between the two versions.

How can I use older Dragonfly scripts written for Python 2.7?

This version of Dragonfly has been written with backwards-compatibility in mind. Older Dragonfly scripts, many of which were written with Python version 2 in mind, will either work perfectly without any changes or will after minor changes.

Although Python version 2.7 reached the end of its life in January 2020, Dragonfly has, for the most part, retained support for it. This has been done because the library has always supported Python 2 and because retaining this support is, at present, neither difficult nor detrimental to the library’s support for Python version 3. The Kaldi engine back-end is the one (optional) component of Dragonfly that requires Python version 3.

If the reader must use Python 3, then the Python 2 code typically needs to be converted. The following two command-line programs may be used to this end:

  • 2to3 - reads Python 2 source code and applies a series of fixers to transform it into valid Python 3 code.

  • python-modernize - uses 2to3 to make Python 2 code compatible with Python 3.

The Python 2-3 code porting guide may also be of interest.

A number of older Dragonfly command modules also include the following code:

try:
    import pkg_resources
    pkg_resources.require("dragonfly >= 0.6.5")
except ImportError:
    pass

Since the distribution name has been changed to dragonfly2, dragonfly will need to be replaced with dragonfly2.

Where are some good resources on learning Python?

If you want to use Dragonfly for flexible computer control or for programming in other languages and don’t have much knowledge of Python, then the following resources from might be of use:

Will Dragonfly add support for speech recognition engine X?

The answer is ‘it depends.’ No new engine implementations are currently under development. Contribution of new implementations is welcome, provided certain criteria are met. Please see Contributing New Speech Recognition Engines for more information.

API Questions

How do I use an “extra” in a Dragonfly spec multiple times?

Sometimes it is desirable to use the same “extra” multiple times in a Dragonfly Compound, CompoundRule or MappingRule specification (or “spec”). Unfortunately, this is not directly possible. But, the special RuleWrap element may be utilized to reference an “extra” multiple times. See the following example:

from dragonfly import (Choice, RuleWrap, RuleRef, Key, MappingRule,
                       Grammar)

# Define a Choice for matching "alpha", "bravo" or "charlie".
my_choice = Choice("", {
    "alpha": "a",
    "bravo": "b",
    "charlie": "c"
})

# Use *my_choice* to make a private rule using RuleWrap.
my_choice_rule = RuleWrap("", my_choice).rule

# Define *Letter1* and *Letter2* as references to the new rule.
alpha_extras = [
    RuleRef(my_choice_rule, "Letter1"),
    RuleRef(my_choice_rule, "Letter2")
]

# Define a command for typing two letters separated by a space.
mapping = {
    "<Letter1> and <Letter2>": Key("%(Letter1)s, space, %(Letter2)s")
}

# Create a new grammar with this command and load it.
grammar = Grammar("letters")
grammar.add_rule(MappingRule(mapping=mapping, extras=alpha_extras))
grammar.load()

Is there a way to re-use a function with different “extra” names?

Dragonfly’s Function action class is normally used to call a Python function when a spoken command is recognized. Function actions pass recognized “extra” values via key word arguments, rather than positional arguments.

Below are two methods to re-use a Python function without redefining it:

from dragonfly import Function

# Define a function to be used by two Function actions.
def add_and_print(x, y):
    print("%d" % (x + y))

# --- Method one ---
# Use a lambda function.
Function(lambda x, z: add_and_print(x, z))

# --- Method two ---
# Use the optional 'remap_data' argument to pass the 'z' argument
# as 'y' internally.
Function(add_and_print, dict(z='y'))

See the Function action’s documentation for more information and code examples.

Is there a way to recognize negative integers with Dragonfly?

Yes. The simplest way of recognizing negative integers is to use IntegerRef and Modifier elements together in a command with an appropriate prefix word such as “negative” or “minus”:

from dragonfly import IntegerRef, Modifier, Text

# Define a MappingRule command for typing a negative integer.
mapping = {
    "(minus|negative) <n>": Text("%(n)d"),
}

# The special Modifier element lets us modify the value of an element.
#  Here we use it to specify the "n" extra as a negated integer between 1
#  and 50.
extras = [
    Modifier(IntegerRef("n", 1, 50), lambda n: n*-1)
]

Is there a way to construct Dragonfly grammars manually with elements?

Yes. The dragonfly.grammar.rule_basic.BasicRule is the rule class to use for constructing Dragonfly rules and grammars manually with elements instead of with compound specs and extras.

The following is an example of how to use BasicRule and common Dragonfly element and action classes together:

from dragonfly import (BasicRule, Repetition, Alternative, Literal, Text,
                       Grammar)

class ExampleRule(BasicRule):
    # Define a rule element that accepts 1 to 5 (exclusive) repetitions
    #  of either 'test one', 'test two' or 'test three'.  These commands
    #  type their respective numbers in succession using the Text action.
    element = Repetition(
        Alternative((
            Literal("test one", value=Text("1")),
            Literal("test two", value=Text("2")),
            Literal("test three", value=Text("3")),
        )),
        1, 5
    )

# Create a grammar with the example rule and load it.
rule = ExampleRule()
grammar = Grammar("BasicRule Example")
grammar.add_rule(rule)
grammar.load()

Please note that extras in action specification strings (e.g. n in Key("left:%(n)d")) will not work for the BasicRule class. For this functionality, a CompoundRule or MappingRule must be used. See the Rules section for more information.

Does Dragonfly support using Windows Speech Recognition (WSR) with the GUI?

Yes. To use WSR with the GUI, specify “sapi5shared” as the engine name.

In a command-module or loader program:

from dragonfly import get_engine
get_engine("sapi5shared")

On the command-line:

python -m dragonfly load -e sapi5shared _notepad_example.py

You may experience problems using Dragonfly and WSR this way, however. The in-process recognizer (sapi5inproc) is more stable than the shared recognizer (sapi5shared).

Is there an easy way to check which speech recognition engine is in use?

Yes. The current engine can be checked using the dragonfly.engines.get_current_engine() function. The following code prints the name of the current engine if one has been initialized:

from dragonfly import get_current_engine
engine = get_current_engine()
if engine:
    print("Engine name: %r" % engine.name)
else:
    print("No engine has been initialized.")

Can I implement my own custom Dragonfly engine externally?

Yes, you can write a Dragonfly engine implementation externally and use it like any of the in-package engines.

Implementing a custom Dragonfly engine is a complex task. It is recommended that you start with a copy of the text-input engine source code and make alterations with reference to the code of other engines. The text-input engine source code may be consulted via the source code links on this page.

If you want to customise say, the Natlink engine, start with the code (or classes) for that engine instead.

Once you have implemented the required engine methods and classes, you’ll need to register an engine instance with the special register_engine_init function:

from dragonfly.engines import register_engine_init
my_engine = MyEngine()
register_engine_init(my_engine)

Your engine instance will then be returned by Dragonfly’s get_engine() function, when it is invoked:

>>> from dragonfly.engines import get_engine
>>> get_engine()
MyEngine()

Your engine implementation should now be useable. Please note, however, that you won’t be able to use it with Dragonfly’s Command-line Interface (CLI) or test suite without a few modifications.

In order to use your engine with these facilities, an instance of your engine must be registered and have a unique name before the get_engine() function is invoked by them. In addition, if you want to test your implementation against Dragonfly’s test suite in a clone of the git repo, you will need to add an entry to the special engine_tests_dict:

# Do this in setup.py around line 59 or in an imported module.
from dragonfly.test.suites import engine_tests_dict
my_engine_tests = engine_tests_dict['text'][:]
my_engine_tests.remove('test_engine_text')  # Replace with test_engine_<name>.
engine_tests_dict[my_engine.name] = my_engine_tests

Troubleshooting Questions

Why are my command modules are not being loaded/detected?

If you have placed Python files into the MacroSystem / user directory (using DNS/Natlink) or the directory where your module loader script is (using another engine) and there is no indication that the files were loaded, then there can be a few reasons why:

  1. Your Python files don’t start with an underscore _ and end with .py.

  2. You’ve put the files in the wrong directory. If you’re using Natlink, then try running the Natlink configuration- program to double check where Natlink loads files from.

In the case that your command modules are being loaded and you’re getting error messages not mentioned in the FAQ, then see the Unanswered Questions section.

How do I fix “No handlers could be found for logger X” error messages?

This error is specific to Python 2. It isn’t a Dragonfly error, but as many Dragonfly users still use Python 2, it is listed here. This is one common example of the error:

No handlers could be found for logger "action"

There are two easy methods for to solving this problem:

# --- Method one ---
# Set up a basic logging handler for console output using the 'logging'
# module.
import logging
logging.basicConfig()

# --- Method two ---
# Set up Dragonfly's logging handler from the 'dragonfly.log' module.
# This sets up a logging handler for console output, appends log messages
# to a log file (~/.dragonfly.log) and sets sane defaults for Dragonfly's
# internal loggers.
from dragonfly.log import setup_log
setup_log()

For either method, add the two lines of code near the top of one of your command modules, or command module loader script, if you use one.

Cannot load compatibility module support error when starting Dragon

This is a known issue with Natlink. Please see this Natlink troubleshooting page for solutions on how to solve this and other issues that occur before the Natlink messages window appears.

How do I fix “failed to decode recognition” errors?

“Failed to decode recognition” is the error message displayed when Dragonfly is unable to match what was said to a grammar rule. One way around this problem to add an “extra” for reserved words:

from dragonfly import Choice, Text

mapping = {
    "reserved <reserved>": Text("%(reserved)s")
}

extras = [
    Choice("reserved", {
        "alpha": "alpha",
        "bravo": "bravo",
        "charlie": "charlie",
    })
]

How can I increase the speech recognition accuracy?

Low recognition accuracy is usually caused by either bad-quality audio input or a speech model that isn’t trained to your voice or use case. You might try the following:

  • Re-positioning your microphone.

  • Using a different microphone.

  • Training words or phrases.

  • Change the speech recognition engine settings (e.g. adjust Dragon’s accuracy/speed slider).

  • Using a different engine back-end if possible, e.g. the Kaldi back-end is typically more accurate than CMU Pocket Sphinx and WSR back-ends.

Dragonfly also has programmatic methods for increasing recognition accuracy. They can be used to fine tune accuracy for specific commands or parts of commands:

  1. Kaldi Grammar/Rule/Element Weights (Kaldi-only)

  2. Quoted words in dragonfly.grammar.elements_basic.Literal elements (Dragon-only)

Why isn’t Dragonfly code aware of DPI scaling settings on Windows?

There can be problems with the mouse action on Windows Vista and above if the system is set up to use one or more monitors with a high number of dots per inch (DPI). For this reason, Dragonfly attempts to set the DPI awareness for the process when it is imported.

If you need to set the DPI awareness manually using a different DPI awareness value, do so before importing Dragonfly. The following is essentially what Dragonfly does internally on Windows 8.1 and above:

import ctypes
ctypes.windll.shcore.SetProcessDpiAwareness(2)  # PROCESS_PER_MONITOR_DPI_AWARE

For more information, please see Microsoft’s own documentation:

How to use Dragonfly actions with administrative applications on Windows?

It is normally not possible to use Dragonfly to interact with applications running in elevated mode. This is a security feature of the Windows operating system. Luckily, there are a few ways around this. They are listed below.

1. Use Dragon’s built-in commands instead

If you’re using Dragon, this is the simplest way. Dragon’s built-in commands can control administrative applications. No additional configuration is required.

2. Load your command modules from an elevated process

If you are not using Dragon, or if you want to use your own commands instead of the built-in ones, you can load them from an elevated Python process. This requires administrative access to the machine you’re on.

The procedure for loading command modules from an elevated process is as follows:

  1. Download or find locally the module loader script for your preferred Dragonfly engine. If you use Dragon and Natlink, this is the dfly-loader-natlink.py example script.

  2. Locate the folder containing your command module files.

  3. Place the module loader script into this folder.

  4. Open a command prompt window as administrator.

  5. Navigate to your command module files folder and run python.exe dfly-loader-natlink.py or equivalent.

Dragonfly’s command-line interface may be used to load your command module files instead of a loader script. The effect is the same.

Please note that, if you’re using Dragon, your command module files should be moved into a separate folder first, so that they aren’t loaded normally by Natlink. If they are loaded normally, your voice commands may not affect administrative applications.

Why aren’t Dragonfly’s input actions working on my Linux system?

Dragonfly’s Key, Text and Mouse action classes use the xdotool program on Linux. These actions will not work if it isn’t installed. It can normally be installed through your system’s package manager. On Debian-based or Ubuntu-based systems, this is done by running the following console command:

sudo apt install xdotool

The Window class also requires the wmctrl program:

sudo apt install wmctrl

The keyboard and mouse input classes on Linux (or similar) systems will only work in an X11 session. The following error will occur if these classes are used under Wayland:

NotImplementedError: Keyboard support is not implemented for this platform!

Wayland users are recommended to either, switch to X11, or to use Windows or macOS instead. You might also consider using numen, a different voice control system with excellent Linux support.

If you are using X11 and still see this message, it means that the DISPLAY environment variable has not been set.

Unanswered Questions

If your question isn’t listed above, then there are a few ways to get in touch: