Ish: Image-Based Chat Room
Ish is a web chat room with a special trick up its sleeve: every word is translated into an image. The name “Ish” is meant to indicate the imperfections inherent in communication, especially when attempting to jump from words to images and back again.
Ish is being actively developed for release as a mobile app.
An Ish user is first prompted for a nickname:
After entering a nickname, the user joins the group of users currently using Ish, and can send messages to the chat room by typing in a message and pressing enter:
The words in each message are translated into images and posted to the chat room as images. Example message:
In the example above, the string of images indicates the sender’s nickname (“KW”), a colon, and the message “This is ish image chat”. Also, the gray box on the right shows the list of current users.
Code in the browser parses the message as it is typed into the message input box, breaks the message into space-separated tokens, and does a Google Image search for each token. When the search results are obtained, the DOM is manipulated to display the images instead of the words. The message is also sent to the browsers of any other users, which do the same lookups and DOM manipulations within their own browsers.
Clients find each other by registering with a Socket.io server which handles group membership tasks, and by send data by registering with a PeerJS server and using peer data connections to send messages that the users want to send to each other. All image lookups are done by the browsers, with no server lookups. All messages pass between browsers directly, with no messages passing to or through the server. Once a browser has associated an image with a word, future uses of that word in a message will not incur additional image lookups in that browser.
The system has the following custom-coded components:
- WebRTC socket server: server.js
These components make notable use of:
Index page: index.html
The index page creates the DOM, which includes some HTML elements (e.g. input, button, div elements) that Ish will manipulate. The elements all have CSS classes and IDs that enable targeting by Ish code.
Important CSS classes/IDs:
- div#ish-name: your user name
- input#ish-name-box: input element for your user name
- button#ish-name-button: button to submit your user name
- div#ish-content: container for chat room content
- input#ish-outbox: input element for your message
- div.ish-window: the messages that have been sent to the chat room
- img.ish-word-SOMEMD5HASH: image elements in a message have these word-specific classes, which allows jQuery to target the elements that correspond to a certain word (by that word’s MD5 hash)
- div#ish-clientbox: container for user list
- ul#ish-clients: list of all users in chat room
Ish library: ish.js
The Ish library, ish.js, contains most of the site functionality. The index page calls the initPage() function, which does the following:
- connect to the PeerJS server and register callback handlers for PeerJS connection events
- send messages to other users
- receive messages from other users
- connect to the socket server and register callback handlers for Socket.io events
- associate a nickname to self, and keep track of nicknames of other users
- join the group of users currently using Ish by sending user nickname and Peer ID to all users
- update each browser’s list of current users whenever a user joins/leaves
- register jQuery event listeners
- perform Google Image lookups as message is being typed
- perform the DOM changes (inserting images) when message is submitted
- send the messages to other users
The Ish library defines many functions to make this happen:
- initPage: (described above)
- updateClients: update the list of clients (the variable and the ul)
- connectToBroker: connect to a PeerJS server, and a Socket.io server
- setName: set nickname in div#ish-name, unhide div#ish-content
- md5Hash: compute MD5 hash of word
- setImage: use jQuery to set the image source of a word’s image to the remote image URL, which causes the browser to fetch the image
- wordToImage: convert a word into an image using the Google Image API
- splitMessage: return an array of the words in a message
- wireMessage: send message out through a peer data connection
- escapeWord: escape special chars in string, so we can use it in HTML attributes
- showMessage: display a new message (div.ish-message) in div.ish-window
- sendMessage: show the message in local browser via showMessage, and send the message via wireMessage
- receiveMessage: show incoming message in local browser via showMessage
Socket.io server: server.js
The Socket.io server (server.js) manages the list of current Ish users, and sends that list to all users whenever a user joins, leaves, or changes a nickname (via Socket.io connection events for “join”, “leave”, “nick”, and “disconnect”).
The list of users is held in an object, ish.clients, which has each user’s PeerJS ID as the key, and each user’s Ish nickname as the value. This object is sent to all users via the Socket.io emit() method.
Future work should include optimization (sending only updates), and purging inactive users from the list periodically.
PeerJS data connections are not completely supported by any browser
PeerJS data connections are supposed to support the sending and receiving of JSON objects. However, browsers do not currently support this reliably, so the only data that can be sent across a data connection is a string. I originally wrote the code to send and receive objects, but had to refactor to avoid the bugs in PeerJS data connections. I’m watching PeerJS – WebRTC status for fixes.
Also, PeerJS support varies greatly between browsers, and appears to behave unreliably even between different versions of the same browser. For example, in testing we saw that users with Chrome 30 might only see messages sent by other users with Chrome 30, and would not see messages sent by users with Chrome 29 or Chrome 31. So cross-browser support isn’t there, and neither is cross-version support.
Google Image Search API query limits
Google Image Search has an API query limit. Sending a long message through Ish might cause some image lookups to fail. I refactored the code to avoid this where possible, mainly by doing lookups asynchronously at the earliest possible time. The refactored code currently watches the message input box, constantly parsing it for space-separated tokens, and doing lookups on those tokens even before the message is completely entered. This also improves the user experience by doing the lookups early, so they appear very quickly once the user hits enter and submits the message.
I also added API rate limit error handling, which queues search queries and retries them after a delay. This error handling is not robust and should be improved.
Google Image Search API v1 is deprecated
Google has deprecated the API that Ish uses. Future versions will likely have to use a different image API, or some other search API (likely secured by OAuth) that Google put out to replace the original Image Search API. It may not be possible to do such a clientside-heavy implementation with an OAuth-secured API.
Google Image Search results often include unavailable images
I found that the top result in an image search is frequently unavailable (404, etc.), which caused certain words to never get translated into images. I refactored the code to do some error handling, which retrieves a lower ranked search result in this case. This error code is not robust, and should be improved.
DOM manipulation kind of sucks
DOM manipulation is confusing and error-prone, even with jQuery. I’m convinced that AngularJS would be a better fit for this application.
Web chat rooms are so 1997
This thing needs to be on the phone, like yesterday.