https://github.com/spiiin/fceux_luaserver
In short - it's a script, that makes NES emulator remote-controlled, and server, that can send commands to the emulator.
This project is also related to my CadEditor project - Universal Level Editor project, and powerful tools, needed to explore games, so I also used my fundition.io project tag.
Why someone needs it?
Several emulators, and Fceux also, can run Lua-scripts for controlling them. But Lua is a bad language for general programming. Basically, it's the simple language for calling C-functions. Authors of emulators use it as a scripting language only for one reason - it's lightweight. Accuracy emulation needs many CPU resources, and scripting is not the primary goal for authors.
But now, personal computers have enough resources for NES emulation, so why not used powerful scripting languages like Python or JavaScript for write script for emulators?
Unfortunately, there are no mainstream NES emulators, that can be controlled with these languages. I know only about Nintaco, it's also has fceux core, rewritten to Java. So I want to create my own project.
It's a proof-of-concept only - it not robust, or fast, but it's working. I created it for myself, but there is the frequent question, how to control emulator externally, so I decided to publish it's sources as is.
Lua side
Fceux emulator already has several Lua libraries embedded in it. One of them - LuaSocket library. It's not a lot of documentation about it, but I found code snippet in XKeeper Lua-scripts collection. It used sockets to send commands from Mirc to fceux.
Actually, code that create socket:
function connect(address, port, laddress, lport)
local sock, err = socket.tcp()
if not sock then return nil, err end
if laddress then
local res, err = sock:bind(laddress, lport, -1)
if not res then return nil, err end
end
local res, err = sock:connect(address, port)
if not res then return nil, err end
return sock
end
sock2, err2 = connect("127.0.0.1", 81)
sock2:settimeout(0) --it's our socket object
print("Connected", sock2, err2)
It's a low-level tcp-socket, that send and receive data per 1 byte.
Fceux lua main cycle looks like:
function main()
while true do --loops forever
passiveUpdate() --do our updates
emu.frameadvance() --return control to the emulator, and it's render next frame
end
end
And update cycle looks like:
function passiveUpdate()
local message, err, part = sock2:receive("*all")
if not message then
message = part
end
if message and string.len(message)>0 then
--print(message)
local recCommand = json.decode(message)
table.insert(commandsQueue, recCommand)
coroutine.resume(parseCommandCoroutine)
end
end
It's not very hard - we read data from the socket, and if next command in it, so we parse and execute it. Parsing made with coroutines - it's powerful lua concept for pause and resume code execution.
One more thing about fceux Lua system. Execution of emulation process can be paused from Lua-script, how it can be resumed from the socket, if the main cycle is paused?
Answer - there is the one undocumented Lua function, that will be called even if emulation paused:
gui.register(passiveUpdate) --undocumented. this function will call even if emulator paused
So, we can pause and continue execution with it - it will be used for setting up breakpoints from a remote server.
Python side
I created a very simple RPC-protocol, based on JSON commands. Python serialiazed command and arguments to JSON string and sent it via socket. Next, it wait to answer from lua. Answers has name "FUNCITON_finished" and field for results.
This idea encapsulated in syncCall class:
class syncCall:
@classmethod
def waitUntil(cls, messageName):
"""cycle for reading data from socket until needed message was read from it. All other messages will added in message queue"""
while True:
cmd = messages.parseMessages(asyncCall.waitAnswer(), [messageName])
#print(cmd)
if cmd != None:
if len(cmd)>1:
return cmd[1]
return
@classmethod
def call(cls, *params):
"""wrapper for sending [functionName, [param1, param2, ...]] to socket and wait until client return [functionName_finished, [result1,...]] answer"""
sender.send(*params)
funcName = params[0]
return syncCall.waitUntil(funcName + "_finished")
So, with this class lua methods can be encapsulated in Python classes-wrappers:
class emu:
@classmethod
def poweron(cls):
return syncCall.call("emu.poweron")
@classmethod
def pause(cls):
return syncCall.call("emu.pause")
@classmethod
def unpause(cls):
return syncCall.call("emu.unpause")
@classmethod
def message(cls, str):
return syncCall.call("emu.message", str)
@classmethod
def softreset(cls):
return syncCall.call("emu.softreset")
@classmethod
def speedmode(cls, str):
return syncCall.call("emu.speedmode", str)
And called exactly how it called from lua:
#Restart game:
emu.poweron()
Callbacks
Lua can register callbacks - functions, that will be called after certain conditions. We can encapsulate this behavior in Python. It needs some additional trick toimplement it.
At first, we save callback function handler in python, and save this callback to lua.
class callbacks:
functions = {}
callbackList = [
"emu.registerbefore_callback",
"emu.registerafter_callback",
"memory.registerexecute_callback",
"memory.registerwrite_callback",
]
@classmethod
def registerfunction(cls, func):
if func == None:
return 0
hfunc = hash(func)
callbacks.functions[hfunc] = func
return hfunc
@classmethod
def error(cls, e):
emu.message("Python error: " + str(e))
@classmethod
def checkAllCallbacks(cls, cmd):
#print("check:", cmd)
for callbackName in callbacks.callbackList:
if cmd[0] == callbackName:
hfunc = cmd[1]
#print("hfunc:", hfunc)
func = callbacks.functions.get(hfunc)
#print("func:", func)
if func:
try:
func(*cmd[2:]) #skip function name and function hash and save others arguments
except Exception as e:
callbacks.error(e)
pass
#TODO: thread locking
sender.send(callbackName + "_finished")
Lua server save this handler, and call generic python callback function with this handler.
After that, in Python, we create additional thread, that checks, if registered callback function need to be called:
def callbacksThread():
cycle = 0
while True:
cycle += 1
try:
cmd = messages.parseMessages(asyncCall.waitAnswer(), callbacks.callbackList)
if cmd:
#print("Callback received:", cmd)
callbacks.checkAllCallbacks(cmd)
pass
except socket.timeout:
pass
time.sleep(0.001)
Last step - callback function do some work and return flow control to lua caller.
jupyter notebook
Open FceuxPythonServer.py.ipynb notebook and run first cell:
Now you must run fceux emulator with ROM (I used Castlevania (U) (PRG0) [!].nes for my examples).
Next, start lua script fceux_listener.lua. It must connect to running jupyter python server.
I do all these things with one command-line command:
fceux.exe -lua fceux_listener.lua "Castlevania (U) (PRG0) [!].nes"
You are able to send commands from Jupyter to Fceux (you can execute Notebook cells one by one and see results).
Full example can be viewed on github:
https://github.com/spiiin/fceux_luaserver/blob/master/FceuxPythonServer.py.ipynb
It contains simple functions:
Callbacks:
And even complex script for moving Super Mario Bros enemies with mouse:
The script is not very robust, it needs additional checks for all data, received from the network. Also, it not very fast. It's better to use some binary RPC protocol instead of text. But my implementation not need compilation, so it can be used "as is". The script can switch execution context from emulator to server and back 500-1000 times per second. It's enough for almost all applications, excepts some special debugging case (for example per pixels or per scanlines PPU debugging, but Fceux does not support it whatever).
Other possible applications:
I used:
Fceux - http://www.fceux.com/web/home.html
It the classic NES emulator, that many peoples using. It not updated by a long time, and not the best by features, but it still defaults emulator for most romhackers. Also, I chose it, because it already includes several Lua libraries, LuaSocket is one of them, so I don't need to implement sockets myself.
Json.lua - https://github.com/spiiin/json.lua
It's pure lua json implementation. I used it because I want to provide the sample, that no need compilation). But I need to create the fork of the library because some other lua library inside fceux code (there are no sources for these libraries) override lua system tostring function, and it brake json serialization (my rejected pull request ) to the original library.
Python 3 - https://www.python.org/
Fceux Lua script tries to open tcp-socket and to listen for commands sent via it. Server, that will send commands to the emulator, can be implemented with any language. I used python because of it's the philosophy of "Battery included" - a lot of modules included by default (socket and json also), and others can be added without problems. Also, Python has libraries for working with Deep Learning, and I want to experiment with it for creating AI for NES games.
Jupyter Notebook - https://jupyter.org/
Jupyter Notebook is cool python environment, with it, you can write python commands (and not only) in table-like editor inside browser interactively. Also, it's good to creating interactive examples.
Also, I used dexpot, because I need to pin fceux window topmost. Windows can't do that by default, and that window manager can. Also, it is free for personal use.
This is proof-of-concept of the remote controlling emulator. It can be used as a base for other projects. I implemented all Lua functions, that emulator has. Other more complex examples can be implemented and committed to the repository. Several improvements for speed can be done also.
Nintaco - Java NES emulator with api for remote controlling
Xkeeper0 emu-lua collection - collection of many Lua scripts
Mesen - C# NES emulator with powerful Lua script abilities (with no embedded socket support for now)
CadEditor - Universal Level Editor and powerful tools for exploring games. I used the project from post to explore games and add it to the CadEditor.
Use it, test it, explore NES games with it. Send me your scripts by pull requests.