I'm Nicholas FitzRoy-Dale, a software developer in Sydney, Australia. I'm interested in embedded systems and operating system research, code optimisation, 8-bit music, rock climbing, shiny things...
Five-minute AOSP hack: remove proximity sensor support
I recently dropped my Nexus 4 and broke its LCD. If replacing this £85 part wasn’t annoying enough, I also managed to lose the tiny rubber part which makes the proximity sensor work. Without it, the LCD turns off, and stays off, during phone calls. (Actually, I didn’t break the LCD — I just cracked the front glass — but I had to buy a new LCD + glass + digitiser assembly anyway.)
Fortunately, the proximity sensor sucked even before I broke it. :) I don’t press phones right up to my face, so talking is often a flickery screen-on, screen-off business anyway. So I hacked AOSP to ignore any hardware-supported proximity sensors.
With this change, you can control the LCD normally using the power button, even during phone (or Skype, etc) calls.
wzdd@ubuntu:~/aosp-nexus4/frameworks/base$ git diff
diff --git a/core/java/android/hardware/SystemSensorManager.java b/core/java/android/hardware/SystemSensorManager.java
index 0204e94..9dd8d7f 100644
--- a/core/java/android/hardware/SystemSensorManager.java
+++ b/core/java/android/hardware/SystemSensorManager.java
@@ -265,7 +265,7 @@ public class SystemSensorManager extends SensorManager {
Sensor sensor = new Sensor();
i = sensors_module_get_next_sensor(sensor, i);
- if (i>=0) {
+ if (i>=0 && (sensor.getType() != Sensor.TYPE_PROXIMITY)) {
//Log.d(TAG, "found sensor: " + sensor.getName() +
// ", handle=" + sensor.getHandle());
fullList.add(sensor);
My Android device (a Nexus 4) won’t let me dismiss the voicemail notification icon without actually clearing the voicemail. This is pretty annoying, particularly since either my phone provider (Three UK) or my device sometimes gets it wrong and tells me that I have voicemail when I don’t.
If you’re compiling your version of Android from the AOSP source code (and I realise this means almost nobody) you can change this by removing the piece of code in the phone app which tells Android not to let you dismiss the notification. Here’s a patch against 4.2.2:
wzdd@ubuntu:~/aosp-nexus4/packages/apps/Phone$ git diff
diff --git a/src/com/android/phone/NotificationMgr.java b/src/com/android/phone/NotificationMgr.java
index 42dcdb6..906ba61 100644
--- a/src/com/android/phone/NotificationMgr.java
+++ b/src/com/android/phone/NotificationMgr.java
@@ -1283,7 +1283,7 @@ public class NotificationMgr implements CallerInfoAsyncQuery.OnQueryCompleteList
if (vibrate) {
notification.defaults |= Notification.DEFAULT_VIBRATE;
}
- notification.flags |= Notification.FLAG_NO_CLEAR;
configureLedNotification(notification);
mNotificationManager.notify(VOICEMAIL_NOTIFICATION, notification);
} else {
Remember to envsetup and lunch. Then you can patch and “mmm packages/apps/Phone”, then copy the new Phone.apk (from out/) to the device.
KeePass NFC: Unlock your KeePassDroid database with NFC
I’m pleased to announce KeePassNFC, a stand-alone application which lets you use a rewritable NFC tag to unlock your KeePassDroid database. This is a follow-up to KeePassDroid: NFC Support (this is a better implementation of that same idea) and Idea: an NFC authentication scheme.
Unlock your password database to your heart’s desire
To set up the tag, launch KeePassNFC:
Here you must select a database, and then, optionally, a keyfile and password. The options are hopefully self-explanatory. Press “Write NFC” and hold an NFC tag near your device to program it, and you’re done.
Now, holding the tag near your device will automatically load KeePassNFC, which will decrypt your password using the data stored on the fob and launch KeePassDroid.
Why two separate apps? KeePassDroid has very few application permissions, which is good security practise. Using two separate apps means not having to give NFC permission to KeePassDroid itself.
How secure is it? Reasonably. The NFC tag stores only random numbers, and the password itself is encrypted with those numbers. So an attacker would have to scan your NFC tag and either steal or root your Android device to get your password. If you suspect that your NFC tag has been read, you can use KeePassNFC to re-write it with new random values, rendering the previous information useless.
Update: A previous version of this blog included a custom version of KeePassDroid. I’m pleased to say that the current version of KeePassDroid in the Play store incorporates the required changes (technical: it supports being supplied a database password via an Intent), so my custom version isn’t required. For the curious: here is the source code for the required changes.
This NFC implementation grew out of discussion on the KeePassDroid forums. You can view and add to that conversation at Google Groups’ KeePassDroid forum.
Just as I threatened in my previous post, I’ve implemented NFC support in KeePassDroid. This means that you can create an NFC tag which decrypts your database, no password required.
You’ll need a spare, writeable NFC tag, such as one of these.
You’ll also need to use my version of KeePassDroid:
Select a database as normal. When you get to the password screen, enter your password and press the new “NFC…” button.
You will be taken to the NFC writing activity:
Select “Write NFC” and place an NFC card on your device.
When you’re done, you should be able to scan your NFC card and go directly to your password database from anywhere in Android.
Details on the security of this scheme are in the previous post. The short version is that it’s okay if your NFC tag gets stolen or copied, and it’s okay if your phone gets stolen, but if your NFC tag gets copied and your phone gets stolen you have a problem.
Note that you can rewrite the NFC tag as many times as you like. Each time you do, a new encryption key is generated. So if you lose an NFC tag, just write a new one and the previous tag immediately becomes useless.
Comments, questions, and bug reports welcomed. This has been minimally tested (on a Nexus 4) and I make no guarantees that it does anything at all. :)
Of course, I didn’t write KeePassDroid. It’s by Bryan Pellin and a lot of other contributors. Official site.
Update 2: A better (and probably, the final) implementation of this idea is now available here.
Update: I implemented this idea. It works even better than I thought because Android lets you auto-start apps when NFC cards are scanned. More info (and APK) here.
I like the KeePassX password manager, but it’s a hassle to enter my password all the time, especially on a phone. Also, I frequently have to enter the password in places with video surveillance. This feels pretty unsafe.
My phone supports NFC so it would be pretty easy to just store the password on an NFC tag. But of course if someone scans the tag they have my password.
Proposed solution therefore is:
Pad the password to 127 bytes (or some other arbitrary amount). Prepend the password length (or stick a 0 at the end or whatever).
Generate 128 bytes of random numbers.
XOR the padded password with the random numbers to produce an encrypted padded password. Store that on the phone.
Store the random numbers on an NFC fob.
This means that I could scan the fob, the phone would XOR the random values with the encrypted password, and then use the result to unlock my KeepassX database.
If someone stole (or scanned) the fob, they would just get random data. If someone stole my phone they would also just get random data (effectively). The key and the ciphered data are the same length, so knowing one cannot possibly help you deduce the other.
Of course, someone could just steal my phone and scan the fob. While that could happen it seems pretty unlikely as my phone and my keys are always in different places. Also they would have to know that this is how I store my passwords, which they wouldn’t unless I were being specifically targeted, which seems unlikely.
Can anyone see any problems with this scheme? If not I might implement it in KeepassDroid.
I’m pleased to release a Vim plugin that I’ve been working on for a little while. File Pirate is a file picker for Vim. When summoned, it pops up a Vim window and displays all files which match the search term you type. Select a result and press enter, and the file is loaded. The focus is on speed of search and loading: looking for files shouldn’t make you wait and take you out of flow.
There are lots of these sorts of plugins already. I wrote this one because I am currently working with a large directory structure (~50000 files) and existing plugins were very, very slow. File Pirate is fast – for 50k files, you shouldn’t notice any delay when performing a search.
Screenshot:
The other interesting thing about File Pirate is that it tries to be asynchronous – searching and indexing happens in the background, so the user interface is always responsive. This is unusual in Vim, which is single-threaded.
It’s known to work on GUI and console Vims on Mac and Linux. There’s lots more information at the github link, so give it a go if you use Vim on large code bases.
I just finished coaxing the HxC USB Floppy Emulator code to run on my Mac. It was a bit annoying, and in the middle of it I was going through my usual Linux guilt complex about how much easier this would be on Linux and how annoying it was that the Mac was so incompatible with Linux.
But then I got over it and realised that it was going to work. This code base written for an entirely different operating system was going to work, and all it took was a bit of build system tweaking. That’s amazing. The hardest part was understanding the HxC code base. *
Now it’s all working and I’m listening to music disks on my Amiga 1200, using an emulated floppy disk drive consisting of an FPGA connected via an FTDI chip to my Mac’s USB port.
That it worked at all is really cool. Thanks to HxC, LibUSB, and FTDI. Your stuff works perfectly.
* It’s mostly great, but, hey, HxC people, if you get an error opening the chip, how about reporting it? I’d even settle for you not reporting it but not saying “Device OK!” in a cheery but totally bullshit fashion on startup. Put it this way: if the device is unplugged, it’s not OK. Yes, I’ll send a patch.
Hardware SID Player: An 8-bit computer for playing Commodore 64 music
I like SID tunes – music files written to be played by Commodore 64 computers. Back when the C64 was released they sounded much, much better than any other home computer. Nowadays, well, they’re an acquired taste – here are some to listen to while you read this post:
The really interesting thing about SID tunes is that they are actually computer programs. Back then, there was no standard way to specify how sounds should be played, so each C64 composer had their own way of doing things. Every SID file you download contains not just a list of notes, note lengths, and effects – the actual music – but also contains a “play routine”, which is a computer program written for the 6510 microprocessor which was the heart of a Commodore 64. That means that in order to play SID files, you either need a 6510 processor or you need to emulate one. All SID players (such as SIDAmp for Windows or SIDPLAY for the Mac) are actually Commodore 64 emulators – or, at least, partial emulators.
The other interesting thing about SID tunes is that the sound chip which plays them (called the Sound Interface Device, which gave its name to the tunes) is a mixture of digital and analogue circuitry. Some people claim that the SID is so difficult to simulate that no software-based SID player could recreate the sound of a real chip. I’m not that discriminating – software-based SID players sound just fine to me – but I wanted to see how hard it would be to get sound out of the real thing.
The rest of this post is about how I made it, and how you can make one too.
Basic theory
To play SIDs using hardware, you fundamentally need three things:
A 6510 processor
A SID chip
RAM to store the song
With the SID tune stored in RAM, it is simply a matter of ensuring the processor, SID chip, and RAM are connected appropriately, then starting the processor and listening to the dulcet tones of the SID. Or so the theory goes.
Unfortunately, these components aren’t super common. The first thing to do was to obtain the chips.
Sourcing the chips
6510 chips are difficult to find. They are a custom version of the popular 6502 microprocessor made specifically for the Commodore 64. Fortunately, it turns out that the majority of SID tunes out there – perhaps all of them – don’t need the special features of the 6510. So I found I could get away with using a plain 6502, which are available on eBay.
The SID chip is a different story. SID chips were made for the Commodore 64, and production of SIDs stopped in 1993. You can find SID chips on eBay, but they are fairly expensive – expect to pay between £10 and £50. Funnily enough, you can sometimes find complete Commodore 64s selling for less than individual SID chips on eBay.
Finally, we need RAM. I chose the 431000 series of RAM chips, because they are readily available on eBay. 431000s contain 128K of RAM, which you’ll note is twice as much as a commodore 64. That’s not a problem: I simply left the highest address line tied to ground, which meant that the upper 64k of RAM on the chip was never accessed and never used.
Designing the circuit
The 6502 processor has an address bus and a data bus. The address bus is 16 bits wide – enough to address 64 kilobytes – and the data bus is 8 bits wide. It is simple enough to hook the processor to the RAM – simply connect each wire of the address bus to the appropriate pin on the RAM, and do the same for the data bus. Make sure to hook the control wires as well, so that the RAM knows whether the processor is reading or writing.
Now the 6502 will be able to read and write to the RAM, but we have two problems. First, how do we connect the SID? And, second, how do we upload SID tunes to the RAM?
Let’s deal with the first problem first: connecting the SID. The basic problem is that we have more than one device which the CPU would like to communicate with – in this case, it’s just RAM and the SID chip. The typical 8-bit solution for this problem is called an address decoder.
My address decoder
The SID chips sits at addresses 0xD400 to 0xD700 in the Commodore 64 memory (as per the C64 memory map). This simply means that if the C64 wants to access the first byte of SID memory, it will set the 15th, 14th, 12th, and 10th bits of the address bus to “high”, and the remaining bits to “low”. Why those bits? Well, D in binary is 1101 and 4 in binary is 0100, so 0xD400 represented as a 16-bit address in binary is 1101 0100 0000 0000.
That means that our address decoder is conceptually like this:
If bits 15, 14, 12, and 10 are set to 1, and bits 13 and 11 are set to 0, then access the SID.
Otherwise, access the RAM.
If you represented this as a logical expression, you could do it like this:
SID = AND (A15, A14, A12, A10, NOT(A13), NOT(A11))
Basically, if that big expression is true, then select the SID. If it’s false, select the RAM.
There are chips which will do this for us. The 74HC04 hex inverter is 6 NOT gates in one chip, and the 4068 is an AND gate with 8 inputs, i.e. if all 8 inputs are true then the output is true. So we simply connect A13 and A14 to two of the NOT gates on the 74HC04, and connect the outputs of the NOT gates, as well as A15, A14, A12, and A10, to the AND gate. We then have two “spare” inputs to the AND gate which I just connect directly to +5V. And voila – an address decoder! We can connect the output of this address decoder directly to the SID and RAM chips! Well, almost – it turns out that the “chip select” lines on the SID chip and on the RAM are “active low”. So we connect the SID chip to the address decoder via a spare NOT gate, and connect the RAM to the address decoder directly.
If you looked at the C64 memory map, you will see that this address decoder isn’t much like a real C64 – in some cases, it will access RAM when it should be accessing a device. It turns out that this doesn’t matter: most SIDs make use of these other devices, so it doesn’t matter that they’re not there. This makes sense conceptually – why would a SID tune need to access stuff related to the display, or the cassette drive? (This isn’t strictly true actually – but I’ll get back to this later.)
Also, this type of address decoder is oldschool even for the C64 era. The C64 used a custom chip for its address decoder, and normally one would use a CPLD or similar. I did it using discrete logic for fun.
Now for the next problem: getting SIDs into the system, and running the 6502.
Interlude: SID upload and timers
We still have three problems to solve.
First is the problem of getting SID tunes into the RAM, as alluded to above. As currently designed, the system doesn’t have any input devices – so there is no way to program it.
Second is related to things the 6502 needs to run. At the very least, it needs a clock signal. The Commodore 64 ran at 1 MHz (or a little bit less in PAL regions) so we need something that will supply a 1 MHz clock. SID tunes themselves tend to be tied to the display’s refresh rate – which means that we also need some kind of interrupt occurring at 50 Hz (for PAL regions) or 60 Hz (for NTSC regions).
I solved all of these problems using a microcontroller – specifically, an 8-bit PIC.
The second problem – timers – is the simpler one. 8 bit PIC microcontrollers have a PWM module which can be configured to supply a 1MHz signal easily enough. This signal can be directly connected to the 6502’s clock input. The other frequency – 50 or 60 Hz – is a bit low for the PWM module, but is easy enough to do in software. We end up with two wires connecting the PIC to the CPU: one to the clock pin, for the 1 MHz signal, and the other to the interrupt pin, for the 50 or 60 Hz “vertical blank interrupt” which is what SID tunes generally use.
The other problem – getting SID tunes into RAM – is a nightmare.
The simplest solution is simply to connect the PIC to RAM. That would mean connecting the PIC to all 16 address bus lines, and all 8 data bus lines. The PIC I chose (an 18F14K50) doesn’t have enough outputs to do this, but that’s not such a difficulty because there are chips which do a serial-to-parallel conversion, such as the 74HC164 (search “74hc164 PIC” for some simple examples). However, there is a much bigger problem with this simple solution, and that is that most 6502s can’t tristate their address bus.
Tristating the address bus
The problem is that we would want the PIC to put information onto the address bus – that is, set some bits to “high” and some bits to “low” as it uploaded the SID tune to RAM (and note that I haven’t actually got the SID tune to the PIC, either – that’s another problem). The problem with doing this – i.e., putting voltages onto the address bus – is that the 6502 expects to be the only thing that can do that. If two devices do it, then it can result in damage to the 6502.
Some processors can avoid this by “tristating” their address bus pins, which means they can essentially disconnect themselves from the bus, putting themselves into a state called “High-Z”. This prevents damage and lets other devices control the bus. However, the 6502 can’t do this. Interestingly enough, the 6510 CAN do it – it’s one of the few differences between the 6510 and the 6502. However, I was stuck with a 6502 and had to work around this problem. I can’t decide whether my solution is good, or very horrible.
My solution uses the PIC to manually clock the 6502 “blind”, supplying a program that instructs the 6502 to write its own SID file to RAM. Here’s how it works:
I connect the PIC to the data bus. This is totally acceptable, because all 6502s can tristate the data bus.
I then use the PIC to start the 6502, raising and lowering the PIC’s clock line so that the 6502 runs for exactly 6 cycles. This is the number of cycles that the 6502 requires to boot up, according to the data sheet.
The first thing that the 6502 does is attempt to read two bytes from addresses 0xFFFC and 0xFFFD. These two bytes are the “reset vector” – an address to jump to to begin executing code when the 6502 is reset. In a Commodore 64, these bytes are stored in ROM. For my computer, I use the PIC to put the special address 0xFF00 on the data lines, instructing the 6502 to begin executing code at address 0xFF00.
The 6502 then dutifully puts 0xFF00 on the address bus lines and attempts to read a byte on the data bus. This is the first byte of code. I use the PIC to put 0xA9 on the data bus. This is the 6502 instruction LDA, which tells the 6502 to store the next byte that it reads in register A.
The 6502 then reads another byte, which is the argument to LDA. I use the PIC to put the first byte of the SID program on the data bus.
The 6502 then reads another byte, which is the next code instruction to execute. I use the PIC to put 0x8D on the data bus. This is the 6502 instruction STA, which tells the 6502 to store the value of register A in memory (you can see where this is going).
The 6502 then reads another two bytes. These form the 16-bit parameter to STA, which is the address to store the value in register A. I put the starting address of the SID program on the address bus.
I repeat steps 4 to 7 until all bytes of the SID program have been stored.
This method is sort of horrible, but it’s also quite neat, because it means that I don’t have to figure out a way to connect 16 lines from the PIC to the address bus (it would have been possible, using shift registers and special chips which can essentially tristate the bus externally, but messy).
The main issue with this method is that the PIC can’t manually clock the 6502 fast enough to do all of this at 1 MHz. In fact, it runs several times slower than that. This wouldn’t be a problem, but the 6502 uses dynamic RAM for its registers – clock it too slowly, and it forgets the values in its registers. Fortunately, the PIC could manually clock the 6502 fast enough that this wasn’t a problem.
The second issue is that I had to add a couple of wires to my address decoder. The new logic for the address decoder looks like this:
If the PIC has requested control, and the CPU is trying to read data, disable both the RAM and the SID.
Otherwise, if it’s a SID address, enable the SID.
Otherwise, enable the RAM.
The beauty of this approach is that the CPU can be reading from the PIC, and writing to the RAM – until the SID tune has been fully uploaded, and then the PIC can essentially disconnect itself from the system entirely.
Playing the tunes
We still haven’t got SID tunes into the PIC. But not to worry – there are plenty of ways to get data into a PIC. One of the simplest is to use the PIC’s serial port. I decided to be a bit fancy about this and use Bluetooth. There are pre-made modules on eBay, selling for about 5 US dollars, which provide a Bluetooth serial interface – hook the serial part up to the PIC, and you can connect from a PC.
Then it was just a matter of writing a custom program for the PIC, defining a protocol for describing the SID tune over the serial interface (including, for example, the speed of playback – 50 Hz or 60 Hz?), and writing a program for my laptop which would upload SIDs via this interface, and the board was almost ready to go.
There was one last thing. SID files don’t play by themselves – instead, they supply an address in memory called the “play routine” which should be called at either 50 Hz or 60 Hz. As described above, I had connected the PIC to the 6502’s interrupt pin, so that the PIC could cause a 6502 interrupt at the appropriate time. My solution here was to write a small piece of 6502 code which I uploaded at the same time as the SID. The code contained a “cold start” routine, run once to initialise the SID, and also an interrupt handling routine, which simply called the SID play routine. (This code is contained in libsid.py avilable below, if you’re interested.)
Circuit design and PCBs
As you can probably imagine, I wasn’t keen to stay with the breadboarded prototype. I wanted real circuit boards. There are plenty of places which will print short-run circuit designs very cheaply. I chose ITead Studio. I designed my board using Altium, and after one false start with the Gerbers, and a long wait, my PCBs came back! It was time to get soldering.
RSIDs and other limitations
The player has a bunch of problems.
The first, and simplest, is that I made a couple of mistakes on the PCB – essentially, I forgot to add a couple of connections. These were easy to fix with wire, but were a bit embarrassing.
The second problem is a bit more serious. There are two types of SID files. The original type, now called PSID (“Play SID”, named after one of the first SID players), contains a play routine as described above, and a bit of meta-data describing the frequency at which the play routine should be called. However, people started to find this format limiting, and so the RSID (“Real SID”) was born.
The RSID doesn’t include a frequency at which the play routine should be called. Instead, it contains a setup routine which programs one of the Commodore 64’s timer chips to deliver a custom playback speed – or it simply uses its own delay loop, or does whatever else it likes: RSIDs assume that the system they are running on is a pretty-much complete Commodore 64. This obviously poses some problems for my SID player.
The RSID problem probably isn’t as bad as it sounds. I suspect that some static analysis of RSID files will reveal that actually they only use a few C64 features. I could write a program which analyses the files and works out what features they use and how they use them, and then convert them into something appropriate for my SID player.
For example, imagine that an RSID starts by programming one of the Commodore 64 timers to interrupt at 25 Hz. It should be possible to recognise this code using static analysis. I could then simply instruct the SID player to run the appropriate code 25 times a second – in effect, converting RSIDs into PSIDs. I haven’t yet attempted this, because there are many, many PSIDs – most C64 game music is PSID, for example.
Source code and Gerbers
All my code is open source and I hereby place it into the public domain. You can do whatever you like with it, but don’t expect that it will do anything at all and certainly don’t expect support. Feel free to email me, though – I’ll help whenever I can.
Code for PIC18F14K50 (including Makefile, C code, an hex file) – should be portable to most 8-bit PICs.
If you do assemble one yourself, please do get in touch as there are a couple of things I haven’t touched on here (most importantly, an amplifier for the SID output) which could otherwise ruin your day.
Please do also let me know if you spot a mistake in this write-up. I’m sure there are at least a couple.
While working with OpenCL, two things came to light that weren’t obvious from the documentation.
The first is that OpenCL kernels are subject to a timeout, which appears to be five seconds.
That’s kind of reasonable – I had wondered how OS X managed to multiplex GPU resources between OpenCL programs and graphics tasks, and it turns out that it doesn’t: it simply can’t use the GPU while an OpenCL program is running. So it puts a five-second limit on the kernel.
The second thing is not reasonable: when Mac OS X times the kernel out, the user interface doesn’t recover. That is, the UI never updates again. The system is totally fine otherwise, as I can connect to it via SSH and shut it down.
This is a nasty bug, as it makes writing long-running kernels quite tedious. Perhaps it’s a Mac driver issue (Radeon HD 6750M in my case).
The solution for development is to avoid the problem by testing the kernel on a computer which doesn’t have a timeout – I’m now using a headless Linux box to do it (apparently no connected display = no timeout on Windows, Linux, and OS X).
And now that I know about the timeout, I can work around it (by ensuring the kernel doesn’t run for longer than 5s, and continously restarting it with new data).
(I have to say that I haven’t tested this thoroughly, just observed that a) a kernel which takes 4s works fine and a kernel which takes slightly longer does not, and b) 5s seems to be a standard timeout for OpenCL kernels)
I’m pleased to announce KeePassNFC, a stand-alone application which lets you use a rewritable NFC tag to unlock your KeePassDroid database. This is a follow-up to 
