From version 1.6 on Android there is an “accessibility service”. It is not difficult to guess why it was created, but we, as people who are striving for unlimited possibilities, are interested in this service from a slightly different angle. Today we will write a program that will allow you to monitor the input in other applications!

Why do you need accessibility service? It allows you to extend the interface of conventional applications to help users with disabilities or those who can temporarily not be able to fully interact with the device. For example, users driving a car, caring for a small child or finding a very noisy party may need additional or alternative feedback interfaces.

In Android there is a standard service of special abilities – TalkBack. If necessary, developers can implement their own. Such services became possible to write from time immemorial (Android 1.6, API level 4), and with Android 4.0 (API level 14) they received significant improvements. Through the “support library” these improvements are realized for devices with API version below 14.

This service allows you to view the description of all windows running applications and get the data typed by the user (except for the entered passwords, agree that logins and text messages are also, in many cases, enough).

In this article, I’ll show you how to easily implement a service to intercept keyboard input.

To do this, you must create an heir to AccessibilityService. In the onServiceConnected connection method, we need to specify an event filter (the AccessibilityServiceInfo class) that will listen to the service. And in the onAccessibilityEvent method, handle these events.
In the application manifest, you should add the lines of the service description:

<service
  android:name=".KeyRecordService"
  android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
  <intent-filter>
    <action android:name="android.accessibilityservice.AccessibilityService" />
  </intent-filter>
</service>

If everything is done correctly, then in the log you can see something like this:

onAccessibilityEvent: [type] TYPE_VIEW_TEXT_CHANGED [class] android.widget.EditText [package] com.android.chrome [time] 113326642 [text] xakep.ru

The AccessibilityServiceInfo class allows you to set filters for specific applications (who said “mobile banks” or “social network clients”?). On the event we need, you can take screenshots.
Quite simply it is done on the rooted device, and if there is no root, then you need to look for third-party libraries to get screenshots from the service.
An example of the code for obtaining a screenshot from the service on a rooted device:

Process sh = Runtime.getRuntime().exec("su", null, null);

OutputStream os = sh.getOutputStream();
os.write(("/system/bin/screencap -p " + "/sdcard/img.png").getBytes("ASCII"));
os.flush();

os.close();
sh.waitFor();

// Then read img.png as bitmap and convert it jpg as follows

Bitmap screen = BitmapFactory.decodeFile(Environment.getExternalStorageDirectory() + File.separator + "img.png");

// My code for saving
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
screen.compress(Bitmap.CompressFormat.JPEG, 15, bytes);

// You can create a new file name «test.jpg» in sdcard folder.

File f = new File(Environment.getExternalStorageDirectory() + File.separator + "test.jpg");
f.createNewFile();
// Write the bytes in file
FileOutputStream fo = new FileOutputStream(f);
fo.write(bytes.toByteArray());
// Remember close de FileOutput
fo.close();

To work the service requires a separate permission, the user gives it in the settings section “Accessibility -> Services”.

Run the settings programmatically:

Intent intent = new Intent(android.provider.Settings.ACTION_ACCESSIBILITY_SETTINGS);
startActivityForResult(intent, 0);

In the old versions of Android (which are still quite a lot in the wild) the user could not be asked. Such rights could be set programmatically:

Settings.Secure.putString(getContentResolver(),
Settings.Secure.ENABLED_ACCESSIBILITY_SERVICES, "pkgname/classname");
Settings.Secure.putString(getContentResolver(),
Settings.Secure.ACCESSIBILITY_ENABLED, "1");

Permissions for the manifest:

<uses-permission android:name="android.permission.WRITE_SETTINGS" />
<uses-permission android:name="android.permission.WRITE_SECURE_SETTINGS" />

Where pkgname is the name of your package and classname is the name of the service class. In new versions of the OS, the consent of the user is indispensable.

Conclusion

From a programmer’s perspective, Accessibility Services allows you to slightly extend the narrow sandbox framework to interact with other applications. The user can simply be asked to give access to special capabilities and thus gain access to data from other programs. Also it is worth to take into consideration the TalkBack service, which sounds all the elements of the screen, taking the description from the contentDescription property. Google gives priority to search engine marketing for applications that are more optimized for TalkBack.

Users, however, as always, are advised to think three times whether to install a new application at all, or, if so, what rights to give to it. Starting with the version of Android 6.0, this can be done more flexibly.