Windows is an old and complex operating system. It’s been around for a very long time, and while it’s been continuously updated and altered, and parts are removed or replaced all the time, the operating system still houses quite a few tools, utilities, and assets that haven’t been updated or replaced in a long, long time. Most of these are hidden in deep nooks and crannies, and you rarely encounter them, unless you start hunting for them.
Most. But not all.
There’s one utility that I need to use quite often that, seemingly, hasn’t been updated – at least, not considerably – since at least Windows 95, or possibly even Windows 3.x. Using this utility is an exercise in pure frustration, riddled as it is with terrible user interface design and behaviour that never should have shipped as part of any serious software product.
This is the story of the dreaded Character Map. I’ll first explain just how bad it really is, after which I’ll dive into the little application’s history, to try and find out why, exactly, it is as bad as it is. It turns out that the Character Map – or charmap.exe – seems to exist in a sort-of Windows build limbo, and has been stuck there since the days Microsoft scrapped Longhorn, and started over.
Down charmap.exe’s rabbit hole
Even though most of you will know what the Character Map is, I’ll give a short introduction anyway. The Character Map is, quite literally, a map of just about any special character you could ever possibly want to use, from basic Latin characters, to common symbols such as the dollar sign or ampersand, all the way to rare mathematical and scientific symbols. It also gives you access to characters from other, non-Latin alphabets and languages, such as Hebrew.
As a translator, I work with all kinds of documents, often filled with special characters, symbols, and sometimes even non-Latin script, even though I don’t translate from languages using them. This has given me a deep hatred for the Character Map in Windows, and here’s why.
The Character Map in Windows 8.1.
The utility’s main window looks normal enough, but it isn’t until you actually start using it that you notice the many, many flaws in the program. First and foremost, you will most likely try to use your scroll wheel or touchpad to scroll through the long grid of characters. However, this is not possible. Even if you click inside the grid to activate it, it simply does not accept scroll wheel input. Instead, the font dropdown menu at the top of the window always has scroll focus. To this day, even after countless years of using the Character Map, I still instinctively start scrolling with my scroll wheel, only to have it jump from font to font.
The utility’s search function is also something from a bygone era. Apparently, searching for a character is something that’s too difficult for ordinary users, so the search function is hidden behind an “advanced view” checkbox. Searching is so very advanced.
You’d think that in 2015, it would have instant search or autocomplete. It doesn’t. You have to enter a search query, hope it’s the right one, and press enter – only then will it search. This means that if you want to find a character, you’ll need to know its exact name – the name Unicode’s developers decided to use – otherwise you won’t find it. Thus, I learned that Unicode refers to â„¢ as the “trade mark sign” instead of “trademark sign”.
Once you’ve finally managed to find your character or symbol, you’ll have to fight the Character Map to actually get your character of choice into your document. The first thing you’ll probably try is to select the character, press ctrl+c, switch to your document or input field, and press ctrl+v. This doesn’t work. For reasons unknown, you first need to transfer the character to an input field at the bottom of the Character Map, and only then can you copy and paste it. You can do this by double-clicking the character, or by pressing select next to the input field. You then need to select the character inside the input field and copy it (either by keyboard shortcut or by pressing copy).
Since this is 2015, instead of using keyboard shortcuts, you may have considered using dragging and dropping. While drag and drop does work, it has a peculiarity in that you can’t just click on a character and start dragging right away; if you do this, you will actually start scrolling through the list of characters (kind of like dragging in Google Maps). In order to drag, you first need to select the character, release the mouse button, and then click on the character again and start dragging.
I can’t even begin to understand the reasoning behind this… Modal dragging? I don’t even know what to call it.
If you happen to need a set of characters or symbols regularly, you’ll have to perform the search every single time, since Character Map does not have a list of recently used characters or user-defined favourites. There’s no logical grouping either, making it even harder to find what you’re looking for if you don’t know the Unicode name.
There are so many more problems. There are small issues, like the spaces before colons in the user interface or all the UI elements being misaligned. There are also bigger problems, like the help button loading an empty Windows Help window that reads “this content is no longer available” or that the window can’t be resized.
It’s dreadful. In fact, it’s so bad I just use Google and Wikipedia to find characters and symbols. It works, but it’s cumbersome. There has to be a better way – and there is, but we’ll get to that later. First, let’s take a look at where it seems to have gone wrong.
Pinpointing where it went wrong
Having established just how horrible Character Map is, the question arises why this is the case. Developer Steven Troughton-Smith, who has virtual machines at the ready for pretty much every Windows release – final or beta – and I dove into the history of the Character Map, to find out how it has evolved over the years. We discovered something quite remarkable.
Character Map has a long history. It started its life in Windows 3.11, with version number 3.10.
charmap.exe’s version/build number in Windows 3.11.
With the move to Windows 95, the user interface got a minor touch-up, but functionally, it’s the same. The version number got bumped to 4.0 (same for Windows NT 4.0), while Windows 98 has 4.10. Windows ME, meanwhile, has version number 4.90, and Windows 2000 – the first NT-based consumer Windows release – bumps it up to 5.0.2134.
charmap.exe’s version/build number in Windows ME.
charmap.exe’s version/build number in Windows 2000.
With the release of Windows XP, the Character Map got its last update: the search function was added, and its version number was bumped to 5.1.2600.
charmap.exe’s version/build number in Windows XP.
There’s a pattern here that Windows aficionados surely have picked up on already: the version numbers for Character Map match the internal Windows version number. The four-digit part of the version numbers for 2000 and XP should match the Windows build number – and for Windows XP, it does (Windows XP was build 2600). Weirdly enough, the build number for Windows 2000 does not match; Windows 2000 is build 2195, while Character Map lists itself as 2134. This puts the Windows 2000 build somewhere between Windows 2000 Beta 3 RC2 (2128) and Windows 2000 Beta 3 (2183).
This pattern seems to apply to most Windows utilities – if you look at Sound Recorder, for instance, you’ll see it follows the same pattern (sidenote: the Windows 2000 build of Sound Recorder does sport the right build number). This makes sense: these are core Windows utilities that are part of the regular Windows build process, and as such, they get the same build number (mostly, as the peculiar exception of the Windows 2000 build of Character Map indicates) and version number, regardless of actual changes to the application and its code.
That being said, you can guess what version number the charmap.exe has in Windows 8.1, right? It should be version 6.3.9600. You’ll be surprised to learn that… No, it actually has a completely different version number: 5.2.3668. This is quite peculiar, since Windows 5.2 is more widely known as Windows Server 2003 (build number 3790). The build number 3668 puts it right between Windows .NET Server Release Candidate 1 (3660) and Windows .NET Server 2003 RC2 (3718). Other utilities do have Windows 8’s version and build numbers – Sound Recorder or Calculator match up nicely with version and build numbers of 6.3.9600.
charmap.exe’s version/build number in Windows 8.1.
What’s going on here? Why did the Character Map suddenly break out of formation? Steven and I developed a theory (well, he did, mostly): we think it’s got to do with Longhorn.
Way back in august 2004, Microsoft announced it was scrapping what was known as Longhorn, to start afresh with a Windows Server 2003 codebase. This would eventually become Windows Vista, the release that, while troubled in its early days, would lay the foundations for a grand modernisation of the Windows NT platform. We theorised that somewhere during that Longhorn Reset, Character Map fell between the cracks.
To test this theory, Steven booted a virtual machine running Longhorn build 4074 (dated May 2004) to check the version and build numbers for the Character Map in Longhorn. Our theory gained some footing: it has version number 6.0.4074.
charmap.exe’s version/build number in Windows Longhorn.
To further confirm our theory, we had to check the first Windows release after the Longhorn Reset. Lo and behold, the Character Map in Windows Vista has version number 5.2.3668 – the exact same Server 2003-era version/build number in Windows 8.1. In the currently available build of Windows 10, build 10130, it also, still, has version number 5.2.3668.
charmap.exe’s version/build number in Windows Vista.
There are several possible theories that could explain what’s going on here. First, it could be that charmap.exe is part of the regular Windows build process, but that some bug prevents the version/build metadata from being updated – a bug Microsoft is unaware of or doesn’t care enough about to fix.
Two, it might be that build/version numbers for applications only change if their codebase has actually been altered; this would imply that the last change to charmap.exe took place between the two mentioned Server 2003 pre-releases. Any possible changes that caused the version/build number to match Windows’ in Longhorn would have been reverted and disregarded as part of the Longhorn Reset. Coincidentally, this could also explain the irregular build number charmap.exe has in Windows 2000.
The third and last theory is the one I like the most. It could be that during the Longhorn Reset, Microsoft recognised what a pile of crap Character Map really was, and drew up plans to replace it with something new for Vista, and as such, excluded charmap.exe from the regular Windows build process, assuming it’d soon be replaced by something new. However, since these were probably quite tumultuous times inside Microsoft, something, somewhere, went wrong – maybe some people got reassigned, some code got lost, whatever – and the intended replacement never arrived, leading to the situation we have now. Again, Microsoft may simply not even be aware of what’s going on here.
In any event, it’s 2015 now, and even the upcoming Windows 10 is still saddled with a Windows 3.x-era Character Map with a rudimentary search function bolted on. Whatever the cause, this is entirely inexcusable, and needs to be addressed. For Windows 10, I deeply, deeply hope that Microsoft will give us a modern, universal version of the Character Map.
All they need to do, is make an identical copy of the wonderful, delightful, lovable, delicious, exquisite, romantic, tasteful, elegant, and perfectly perfect Character Viewer in OS X. Okay, I’ll grant my perception of it is coloured somewhat by my horrible experiences with the Windows Character Map, but just look at it – it’s got instant search, logical grouping, recently used list, favourites, access to emoji, and it’s easily invokable from both the menubar and by pressing ctrl+cmd+space while in any text input field.
Perfection, I have laid my eyes upon thee.
Microsoft, you’ve got work to do.
When you start replacing all body parts in a human being with superior robotic parts, at what point does it stop being a human being?
For Windows, I believe when they replace the character map application, Windows will stop being Windows.
Thom, you must be the only translator on earth not using Popcharwin (http://www.ergonis.com/products/popcharwin/)
Your points are valid, but also moot…
Or you can remember their keyboard shortcuts which will be much faster. ALT-225, ALT-132, ALT-149, etc
Select the character? Are you sure? I don’t think so. Copy button just works for me.
Character map was designed to build up a group of characters, so this is why it’s quirky. Also, double clicking seems reasonable to me – it was the first thing I did when I just opened the app for the first time in a very long time. I’ve always used it to type a word.. you’d type the parts of the word you can type normally in to the text box, so say: “el capit”, then use the app to find and type “á”, then type “n”. Press “copy” button (you don’t need to select the text first) and paste it in to your document. If you want any more than that, using a keyboard mapping wins over the character map app every time. “el capitán” <– there, I just did it for you to verify what I’m saying is true.
Honestly, whenever I type in any language for more than a few hours, I install the character map. Japanese works extremely well, Swedish on a British keyboard sucks a little as it screws with the layout, but Programmers Polish is extremely easy to get on with as it uses pretty much the standard layout. This won’t work for everyone, but for me it’s way superior and addresses your complaints without needing a new app.
Web alternatives:
http://unicode-table.com/
http://www.fileformat.info/info/unicode/char/search.htm
Linux and Java apps that fix all the mentioned problems except for looking nice:
http://fsymbols.com/character-maps/linux/
I guess Microsoft simply does not care because anyone can simply use a better free alternative anyway.
Hehe, that rationale never stopped them before.
Mjeh.
https://www.youtube.com/watch?v=D3-vBBQKOYU
As I see it, the whole “scroll input goes to the focused element” paradigm is broken.
I much prefer the X11 approach where mouse clicks (the scroll wheel is buttons 4 and 5 internally) go direct to the window under the cursor unless intercepted via XGrabButton() and the window manager can choose whether or not to shift focus as a side-effect.
Having “mouse input goes to the window under the cursor and focus may or may not follow” just feels much more intuitive as a default and it’s definitely more streamlined.
(eg. I’ve got it set up so buttons 1 through 3 switch focus, the scroll wheel doesn’t switch focus, and the back/forward buttons on my Logitech G3 are prevented from reaching apps and instead switch workspaces.)
Agreed. This is window managers behaviour. I too much prefer the usual Linux behaviour. Tweaking this behaviour in Linux window managers is easy, in Windows is it very hard
Off topic,
but I wanted to say what’s up I love OSNews even though I’m not that popular here according to my profile 😉
I was in the Ars AV forum and i made a post about signal chain with digital audio and they straight banned me! I have been reading that site for 15+ years and they cut me for a factual post. I didn’t swear or battle anyone, they just didn’t like my point of view, I guess.
Crazy. Kinda hurt my e-feelings and reminded me of an early lesson told to me, so I had to put it together here
http://wfnk.com/blog/2015/06/why-are-tech-sites-anti-audio/
Back to OSNews regularly scheduled stuff. (Thom – more coverage of audio?)
[I had to post this here because there’s very few sites I go back nearly 20 years with. What happened to reply and take down tactics, or straight up admit when someone makes a point? I was on the internet before most, give the elderly some respect haha]
Well, I think your’re popular here. But I disagree with your audio point of view. Such is the internet that bans happen. I’m ticked that no one will put up with my political arguments. No body likes what I have to say there. Oh well. One of these day’s I’ll just create my own site that everyone will hate.
i didn’t know there could be a point of view in audio, but i know i have plenty of opinions. i don’t get into arguments on pro-audio sites, i try to learn and lurk. it’s when the topic is consumer audio that fights break out. considering how confused the modern consumer is about audio, that’s no surprise.
i just preach signal chain, which says nothing can be better than what came before it. it can only degrade. to truly improve you must start at the top of the chain.
that said — the very end of the signal chain is room treatment and speaker placement and those mean the world, are oftentimes free, and most people completely ignore them.
playback signal chain = 1-source to 2-DAC to 3-amp to 4-output to 5-transport to 6-speaker to 7-room
if you have really good 1,2,3,4 nearly every 5,6,7 sounds good, at least the best it can sound.
As far as digital resolution, I went from 1400k bitrate in 1990 down to 192k in 1999 then to 256k in 2004 and 320k in 2010,
Then I remembered 1400k and heard albums I love at 5400k and was sold. My mastering engineer had been telling me about 24bit for years, and now we all have hardware fast enough to upgrade.
And what has any of this got to do with Windows character map?
Perhaps you would find yourself a little more popular if you didn’t comment off topic rants in threads.
correct…. i’m not so patiently waiting for relevant stories to contain my rants.
Could you clarify and cite the “known psychological issues with ABX testing” that you mention? …because I’d never even heard a hint toward there being a problem with it.
(Though, unless a better alternative can be devised, I’ll stick with ABX as the final arbiter. I’ve seen far too much harm and error brought about in all sorts of fields by trusting one’s ability to be objective.)
I definitely agree with the signal chain part though. Far too many people get crazy ideas about what affects audio quality because they missed a quality bottleneck somewhere up the chain.
(Admittedly, though, that’s also why I have yet to be convinced that 24-bit audio as a release format provides any significant benefit. I’ve yet to see a properly controlled test comparing 24-bit audio to 16-bit downsampled from the same source with proper precautions taken to ensure no glitches crept in due to different decoding paths or poor-quality downsampling.)
I can’t point to anything already compiled that summarizes issues with current listening tests. I’m not sure if anyone has published the proper take down in a simple to understand form. I have read various takes and opinions burried in discussions. I can make a quick list of problems in AB tests that are ignored and could be addressed to yield better data:
1 – Our earbrain cannot do instant recall of the previous quality, relying solely on memory of previous quality while the current version is playing.
2 – Our earbrain immediately reacts negatively when the material is stopped, switched, or skips. This is an interruption to the song and this greatly skews data.
3 – Our natural listening state of music is emotional, not technical. When listening for sound quality we are not listening in a normal state and we are not listening to enjoy, therefore we can’t test the pleasure of the sample.
4 – Volume and brightness will present as higher quality to many, at least initially.
5 – Most sound quality improvement is in the “air” – space and timing of the music. Because of this, often times we hear a quality change in the introduction of the song, not in the middle of the first verse. Also the “breakdown”, the quiet part into a big ending – show details of quality. Therefore fast switching should not be allowed, and half-songs should be the smallest unit of sample.
6 – We are always comparing sound to our own personal listening experiences. If given strange gear we compare it to our own, we seek familiarity. If listening to strange material we aren’t comfortable no matter what, it is not a natural test.
OK gotta wrap it up, I think if you google criticism of AB listening tests you might find some stuff. I support AB tests for most sciences, but when trapping for musical enjoyment it fails for several reasons.
Thanks. I’m currently focused almost exclusively on exams but I’ll put that on my TODO list for afterwards.
What I originally wrote was a very nice way of saying you’re wrong, but generally a nice person otherwise. We just won’t agree on that topic.
Fair enough
So – how does the Longhorn charmap look? Identical to the current one?
Edited 2015-06-17 22:33 UTC
The version number is clearly stuck in time for some odd reason, but if you look at the binary itself, you’ll notice there have been changes. The checksums on 8.1 and 10 differ, for example.
It’s possible it’s getting recompiled for every release, but the build numbers aren’t getting updated. I don’t know about the MS tools, but compilers sometimes embed a timestamp somewhere in the output, resulting in different checksums for otherwise-identical builds…
Fortunately I’ve never had to deal with the Windows Character Map. Looks like I have yet another reason not to switch to Windows.
It also means I can be a bit more critical of the OS X Character Viewer. Don’t get me wrong – it works great most of the time – but as someone who uses it almost every day I feel I have the right to complain a bit.
I don’t know why but for some reason I’m utterly crazy about writing systems. Full Unicode support, the Character Viewer and fonts for several exotic writing systems were some of my favourite new features when I switched from Mac OS 9 to Mac OS X. The ability to search for Chinese characters by radikal and strokes come in handy when I tried to learn Mandarin a few years later.
However, it does feel like the Character Viewer has gotten slightly worse with each redesign.
1. It used to stay open until I closed it. I could also hide it temporarily by minimizing it to the Dock. Nowadays it can no longer be minimized and it seems to be tied to a specific application. Switch applications and the Character Viewer disappears. Switch back and it might reappear if you’re lucky. Be prepared to press ctrl-cmd-space a lot.
2. Scripts used to be grouped by geographic region making them easy to locate. Nowadays one has to select which ones to display in the list on the left. The answer in my case is all of them (except emoji). So, after clicking 100 individual checkboxes I end up with a huge unordered list where I can’t find anything. I can search for individual characters but not for a category or script. To add insult to injury, the checkboxes one uses to customize the list are grouped by geographic region.
3. Where is the font explorer (or whatever it was called)? I used to be able to list all glyphs in a specific font. This was very useful when dealing with scripts that had not yet been included in Unicode and were still using their own weird ad-hoc encodings and custom fonts.
“non-scrolling with scrollwheel”. Annoying and strange, although once you click a character the font-scrolling at least stops. But who would scroll through such a list hoping to find something anyway?
“advanced view”. Also strange that this is needed, but at least it remembers this selection so after your first use it will always be “advanced”
“search”. just searching for part of a name works, so “trade” would find your trade mark sign so would “mark” or “sign” but among way too many other options. try finding “at” for a better example
and then I am going to have to agree with all you said
* no ctrl+c / ctrl+v?
* no drag/drop? (yikes for that modal dragging)
* no recently used (even Words “Insert Symbol” has that)
It does seem to have grouping, although I will not try to pretend that I understand what “Ideographs by Radicals” are (my Japanese wife though…)
I have no problem with the spaces before colons, that actually looks more readable (though non-standard)
All the UI elements seemed decently aligned to me until I started to look more closely
Didn’t you forget the biggest problem though? When you select a character it shows a zoomed in version that blocks the view of the nearby characters and doesn’t go away when you press the ESCAPE key.
[quote]Whatever the cause, this is entirely inexcusable, and needs to be addressed.[/quote]
Why? I have asked around and nobody ever used this feature. I didn’t even know it existed myself and thought you were talking about “insert symbol” in Word (probably even more horrible for anything but the simplest things)
[quote]For Windows 10, I deeply, deeply hope that Microsoft will give us a modern, universal version of the Character Map…. it’s easily invokable from both the menubar and by pressing ctrl+cmd+space while in any text input field.[/quote]
“modern, universal” refers to apps and apps can’t have system integration because of their sandboxed nature. Or in other words…don’t hold your breath
No idea why I spend part of my night on this, but you are right…that thing is quite horrible…like most software that hasn’t been updated in 10 to 15 years (by the way, that strongly supports theory 2: no update means no versionnumber change)
Bonus: Let’s see if you can find and explain “because”
Edited 2015-06-17 23:57 UTC
Linux seems to have the best method for entering special characters, at least in my opinion. If you set this command to run on startup: “setxkbmap -option compose:ralt“, it allows you to enter special characters by holding down the right alt key and typing a two-character combination. For example, to enter á you would hold down right alt and type ‘a or a’. This is obvious and easy to remember (as opposed to Windows Alt-codes). Other common characters:
Type: Result:
`a – – – Ã
‘a – – – á
“a – – – ä
oa РРР̴
^o РРР̫
/o – – – ø
ss – – – ß
oc – – – ©
or – – – ®
PP – – – ¶
It also handles capitalization, so ‘A gives you Ã. No need to open a character map!
Does anybody know if there is a way to do this in Windows as well. My online searches have turned up nothing.
That is only for simple special characters though. Most (software) keyboards handle that automatically. I have 2 keyboards installed on my machine:
* English (United States) – US for development where ‘a becomes ‘a
* Dutch (Netherlands) – United States International for text typing where ‘a becomes á.
Of course the latter also works for some other common special characters:
`a = Ã
^A = Â
“a = ä
‘c = ç
~n = ñ
switching between those software keyboards is done with winkey+space and gives a nice visual indicator. This is all that I need normally
Those are bad example. Accents are already handled like that by most keyboard layouts.
Better examples are oe -> œ ae -> æ mu -> µ
Allow me a short addition:
A character map might still be useful when you do not know how to input one character (or a few ones) you hardly use, “for the overview”. In daily work where you need to input those characters regularly, a character map application is probably simply overkill (text production flow interruption, distraction, “mental break” / paradigm break)…
Like pretty much all Microsoft products, it has gone downhill after Windows XP.
Must have something to do with the quality of the people there. A new generation took over.
It’s clear that Microsoft won’t/can’t update Character Map… They don’t want another EU antitrust case brought against them by the other big Character Map tool companies.
Thom, you can copy all the needed characters to text file and copy from it. Sure it’s clunky, but it’s better than having to deal with an unintuitive interface.
Of course, the better option would be to use alternative software, as some have suggested before.