Factorio-A11y (pronounced "factorio ally") is an accessibility mod and associated voice control grammar for Factorio, which aims to make it possible to play the game with a little as possible input from mouse and keyboard while preserving the spirit of the game.
Paired with voice freely available voice recognition software, you can say:
refuel everythingto put the best fuel you have into every furnace, car, burner inserter/mining drill in your reachrun there, grab stone furnace, click, refuel itto run where the cursor is, get a stone furnace from your inventory, build it, and load it up with the best fuel you have.mine here repple five, craft ten wooden chestto mine the 5 closest trees and craft ten wooden chests
A11y aims to be compatible with other mods and to preserve existing gameplay mechanics; e.g. although it adds a way to move the character via the mouse cursor, rather than having the character instantly teleport from point A to point B, the character still needs to walk there along a valid path and brick/concrete still provide relevant speed boosts.
Currently the mod is done enough to use for the early game; the main thing missing is support for combat. Star this repository if you're interested to help me prioritize my projects.
- Factorio-A11y
Factorio-A11y expects you to use voice control software; as of early 2019, the main options available are:
- Dragonfly (Windows, some Linux support too) - supports Windows Speech Recognition, Dragon Naturally Speaking 13 or higher, or (experimentally) the open source Sphinx speech recognition system.
- Vocola (Windows) - requires Dragon Naturally Speaking 13 or higher.
- Talon (MacOS) - has a built in speech recognition engine, but can make use of Dragon Naturally Speaking 6.
You only need to set up and install one of these.
If you're interested in this mod for serious use, you might also want to check out:
- Vertical mice, graphics tablet pens and trackballs might be less painful to your hands.
- JoyToKey or Gopher360 can remap controllers/joysticks to send keypresses and/or mouse movements/clicks
- eViacam (free, open source, Windows/Linux) can use a standard webcam to control the cursor by tracking your face so that the cursor moves when your head moves
- Precision-Gaze (free, open source, Windows) is better, but requires you to buy eye or head tracking hardware (usually available for < 200 USD).
- Talon (free, MacOS) requires you to buy eye tracking hardware (usually available for < 200 USD), but is probably the most advanced option available (also supports making mouth noises to click).
If you can use a mouse fine and just want to use voice control for convenience, then you can ignore these. (Although it is very cool to control a game using only your eyes!)
- Install the mod into Factorio: follow the setup instructions in
mod/README.md - Install a Factorio voice grammar (a set of voice commands for some software):
- If you use Dragonfly, follow the installation instructions in
dragonfly/README.md - If you use Vocola or Talon, you'll have to first port the Dragonfly grammar over. Send a PR when you're done!
- Start controlling Factorio by voice!
- Start by saying
data reloadto get the voice grammar to know about all the Factorio items, recipes, etc loaded into the game.
A voice grammar is a collection of a set of voice commands and associated logic.
Factorio-A11y assumes you already have a grammar for regular clicking, right clicking, hitting individual keyboard keys (to open/close inventory/map/etc), etc.
You should also know Tutorial:Keyboard shortcuts and TIL all the keyboard shortcuts.
With that out of the way, Factorio-A11y provides the following commands:
The voice grammar loads data from Factorio, but you need to ask it to do so:
data reloadwill force Factorio to dump data which the voice grammar will then readdata list itemswill list out names of known items (things you cangrab)data list recipeswill list out names of known recipes (things you cancraft)
Basically you should say data reload anytime after unloading & reloading your voice grammar, after updating Factorio, or after changing your Factorio mods.
explain itwill print out the name of what you're holding or hovering over, which is very useful in combination with other commands.
run therewill run your character to where your cursor is on the screen.run <direction>will run your character in the given direction indefinitely (useful for exploring)directionis any 1-2 ofnorth,south,east,west
stopwill stop your character from running in a direction indefinitely.
mine herewill mine the closest resource in reach which the player can mine (ore, tree or rock)
raze herewill mine the closest building in reachraze itwill mine the item being hovered as long as it is in reach (building, resource, vehicle, etc)detile herewill mine the tile being stood on (bricks, concrete, etc)
A11y can build ghosts for you thanks to your character's newfound willingness to do hard backbreaking labor:
labor herewill start your character laboring if there are any ghosts in reach: your character will run to each ghost in turn and build it using an item from your inventory. More expensive buildings take longer to build, of course!
Tips:
- Issue a run command to stop the laboring (or manually walk in any direction).
- Ghosts queued for laboring will have a white circle overlaid.
- Ghosts you don't have the right items for will have a brown circle overlaid.
- If you run out of items necessary to build any ghosts, laboring will stop.
Building lines of immediately adjacent buildings (e.g. transport belt lines or rows of assembly machines) is very common so A11y augments building to improve this:
- When you build anything, you'll see a cross-guide showing where adjancent buildings can fit, to help you line up buildings in a line
- Normally shift-clicking will place a ghost of the item you're holding, but with A11y, if you shift click on the cross-guide (while holding a building that's lined up with the guide), you'll place a row of ghosts of those buildings.
craft <count> itwill craft the item being held or hovered over<count>times.<count>is the number of times you want to craft the recipe which creates the hovered item
craft <count> <recipe>will craft the item which<recipe>produces<count>times.<count>is the number of times you want to craft the recipe<recipe>is the name of an item you can craft- For example:
craft 2 transport beltwill craft two regular conveyor belts.
copyandpastewill Shift + Left/Right Click respectively (and hence work as per Manipulating Entities).red <rotate_count>andwrap <rotate_count>(where<rotate_count>is an optional number from 1-4) will rotate/reverse-rotate an entity 1-4 times.- For example,
red 2will reverse the facing of a building.
- For example,
transferandsplitwill Ctrl + Left/Right Click respectively (and hence work as per Manipulating Entities).count <item>will print out how many of that item you have, and let you know how many more you can craft<item>is the name of an item you might or might not have in your inventory- For example:
count wooden chest
grab <item>will transfer that item from your inventory to your cursor (so you can build it or put it into an assembly machine, etc)<item>is the name of an item you have in your inventory- For example:
grab iron platewill put a stack of iron plates into your hand.
vacuum <count> <item>will pick up items of that type from the ground & belts to your inventory<count>is the maximum number of items to pick up (default is 100)<item>is the name of an item lying on the floor or on a belt within your reach- For example:
vacuum 10 electronic circuitwill move up to 10 basic circuits from the ground & belts into your inventory.
NB: for convenience, if you're holding a ghost of an item and your inventory gains one of those items, the ghost will be swapped out for the real item.
You can add fuel (wood, coal, etc) into buildings (burner miners, burner inserters, stone furnaces, etc) and vehicles (cars, tanks, trains):
refuel herewill refuel the closest refuelable entity in reachrefuel itwill refuel the entity being hovered over (building, vehicle, etc) as long as it is in reachrefuel everythingwill refuel everything that is reachable
Tips:
- The highest energy value fuel is used first (e.g. coal before wood).
- Acceptable fuel for each entity is calculated dynamically, not hardcoded, so this should work with mods.
You can load ammo into turrets (gun and artillery turrets) and vehicles (cars and tanks), as well as into your character's guns:
reload herewill reload the closest entity that takes ammo and is in reachreload itwill reload the entity being hovered over (building, vehicle)reload everythingwill reload every entity that is reachable (not including the character's guns)reload selfwill put ammo into your character's guns
Tips:
- The most expensive ammunition is used first (e.g. piercing bullets before regular bullets).
- Acceptable ammo for each entity is calculated dynamically, not hardcoded, so this should work with mods.
- Can we see what someone is hovering the UI? Would be useful to "what is" a hovered item in inventory.
- When vacuuming up ammo, put it in your active firearm first (if it fits)
- Make voice grammar detect Factorio script output location? Or at least read it from config file.
- Voice grammar should save and load last data loaded from Factorio and restore it upon grammar reload
- When mining nearest resource, prefer mining things that collide with the player (e.g. trees & rocks) for convenience in clearing a path.
- Have a way to mine everything in range quickly (for clearing trees)
- Repairing items, using repair packs (ideally it should take some time to repair something, like in the base game)
- Allow aliasing virtual items when crafting or grabbing? E.g. "craft/grab electric"
- oil is not mineable, so should be filtered out from the mining UI. Is there a generic API for detecting non-mineable objects? Looks like the player prototype has a mining_categories field: https://wiki.factorio.com/Prototype/Character and it defaults to
basic-solidaccording to a data raw dump I found. - Mining resources and buildings should take some time - implement the mining hardness formula for this based on the FF post.
It's important to have a uniform interface to perform actions on game entities, so here are thoughts on voice grammar design to inform the API of A11y:
- Most actions should be of the form
<action> <target>. - Actions are things like "mine", "craft", "grab", "run", etc
- Certain actions might have numbers after them - e.g. "craft 15"
- Targets should link to game objects/entities. To support both eye/head-tracking and voice-only
playstyles, we need to support several targeting mechanisms:
it- whatever cursor hovers over (or chose using a selection tool)here- target closest entity eligible for actionall- target all entities eligible for action<item prototype name>- target the entities whose prototype is named this. E.g.grab copper plateormine iron ore<item prototype group>- some items are part of the same named "group" (game.player.selected.prototype.subgroup.name), like all trees aretree.grid <coordinate>- to enable true voice-only play, we need to be able to move and interact via something like a grid system, or naming tiles via tiny UI.
Open questions:
- should mining resources use different keyword than deconstructing buildings? They're the same as far as the game is concerned, but it might be helpful to allow mining items
- there needs to be a name for mining tiles, separate from mining resources
- we have an indicator for the closest resource, and closest building still needs implementing.. but what about tiles? Since the only interesting thing to do with a tile is mine it, maybe this should only be drawn when there's a tile in mining range?
- should the grid system use rows & columns like B3 or should each tile just have its own number? how do these scale for longer reach or interacting outside reach? Due to latency we want to avoid a dragon-mousegrid-like system.
- sometimes you're okay with your character moving to fulfil the command, sometimes not. Should there be
a modifier suffix, like
<action> <target> [<modifier>], where you can say things likemovingto allow your character to move?
-- convert a table to a string
-- see https://github.com/pkulchenko/serpent
game.player.print(serpent.block(p))
-- alternative
game.player.print(inspect(p))
-- drop a traceback
debug.traceback()
-- writes to the game's log
log()
-- writes to stdout (run the game from a terminal)
print()
https://lua-api.factorio.com/latest/index.html
https://wiki.factorio.com/Data.raw https://wiki.factorio.com/Prototype_definitions https://wiki.factorio.com/Prototype/Decorative https://wiki.factorio.com/Prototype/SimpleEntity
https://github.com/Afforess/Factorio-Stdlib/tree/master/stdlib http://afforess.github.io/Factorio-Stdlib/index.html
Thank you to:
- Factorio stdlib - thanks for making Factorio's API much easier to work with.
- rxi's json.lua - thanks for writing a good json library.
- Font Awesome - this library contains icons modified from Font Awesome, licensed under CC Attributions 4.0 license (https://creativecommons.org/licenses/by/4.0/).