Copying a twitter thread[1] here for more explanation
I built CandyGraph a few months ago to scratch an itch and never really got around to talking about it, but it's starting to get some use now, so here it is: https://github.com/wwwtyro/candygraph
I wanted a plotting library that was designed to not only plot a zillion points very fast, but also deal with changes to the presentation of that data at interactive framerates. If I zoomed in on the plot, for example, I wanted the axes to update fast enough that I wouldn't have to worry about the hit. If I changed an axis from linear to log, the rendered data would update immediately; I didn't want the CPU to churn through a million data points and ship them to the GPU again before it rerendered.
So CandyGraph was born. It adopts D3's elegant concept of scales, but it implements them on both the CPU and GPU. When data is rendered, the scaling takes place in the vertex shader. Changing a scale wont cause a missed frame.
On top of scales sits coordinate systems, also implemented for the CPU and GPU. These combine multiple scales into a system capable of dealing with 2D points. It has cartesian and
polar coordinate systems, currently.
It renders everything on the GPU, even axes and text. No need to overlay your speedy WebGL plot with an SVG or another canvas for your axis rendering, and no need to take a perf hit for doing so. And of course, the data rendering itself is speedy; it's all instanced by default.
Finally, CandyGraph comes with a small but growing collection of optimized primitives and higher-level objects to simplify common plotting tasks.
> It renders everything on the GPU, even axes and text.
Yeah, DOM needs immediate mode graphics. Like the one I've added to Sciter, this for example:
element.paintForeground = function(gfx) {
...
}
allows to custom draw a) on top of default element's content (note: any element, not just <canvas>), b) with the paint frequency (60 FPS) and c) on GPU directly - gfx there is the same graphics that HTML content uses.
So you can still use Context2D for GPU drawing without the need for WebGL.
Even you can push vertices on WebGL fast enough there are other problems, text measurement and rendering for example.
WebGL simply have no means for drawing texts.
So I've added Graphics.Text - attributed text layout that caches chars-to-glyphs and layout calculations...
Thanks for the great post about instanced line rendering! I implemented it recently and it works beautifully.
As I commented here https://news.ycombinator.com/item?id=27260950, there's a possible performance improvement to be had from moving away from instancing, as that's apparently relatively slow for low numbers of vertices.
Awesome project! I'm curious how you draw lines? I found that drawing nice lines is quite painful in WebGL; I ended up using the "Screen-Space Projected Lines" technique here:
Maybe a stupid question, since I am not a web dev, but how exactly do you render the text with the GPU? Text rendering ain't simple. On top of that, how does that interact with the browser? Can the browser still select the font (size) of the rendered text? Can the user select it for copying?
For the latter questions, WebGL is rendered as an opaque bitmap in the browser (canvas element) so it amounts to an unselectable image as far as the user is concerned: It's all pixels and no text.
It’d be a fair bit of work, but from a technical point of view you could implement a click and a drag event handler that “selects” the text, and intercept Ctrl+C etc and place the text on the browser clipboard.
Ultimately it would probably feel quite unnatural compared to the native text selection of the browser. And probably would not work very well except in a small subset of browsers.
I think at some point in the future browsers and/or operating systems will ship with machine learning powered systems for recognizing text in bitmaps and for letting users copy recognized text. This is already possible today both with third party non-ML powered OCR software and with third party ML-powered software.
The new Google docs uses a canvas. The feature is not rolled out fully yet but there is a demo document attached to the announcement. The canvas demo document is fully selectable, it feels native. I assume it works as you described, so it's possible to make it feel right, with enough effort to detail.
By placing the right elements at the right place, you can have native input and text selection, but that’s not just WebGL anymore; That’s a regular (but transparent) webpage with a WebGL background that mimics your input. It takes quite a bit of effort to emulate that effectively since there’s no guarantee that the visible text rendered by WebGL renders with what the browser renders outside it. Probably the 80/20 rule applies here.
> On top of that, how does that interact with the browser? Can the browser still select the font (size) of the rendered text? Can the user select it for copying?
This looks promising. I just created a visualization of 5M data points with Observable Plot and found it easy to make the plot aesthetically pleasing — there are a lot of examples, the built-in color scales are lovely, etc. But the browser really struggles to filter/summarize/render the data.
On your Examples page, I would recommend making it more clear that these are live, buttery-smooth WebGL canvases. I thought I was looking at screenshots until I accidentally moused over the Health and Wealth of Nations plot. It would be more convincing to, say, put the scatter plot first, animate the points, and show the framerate. Give me a button to add 10^n points at a time and see how it performs.
This does look promising: responsive without a cumbersome license. What I would like to see is a 3d scatter plot that can handle up to 100K points. The view would not be limited to, and I hope I am describing this right, a "static camera angle with the viewed center of mass being rotatable". I'd like to be able to fly/zoom in to near a dot and click on it for more information. Ideally you could put it on github.io and accept user files of the form : x,y,z coordinates in first 3 columns and a description in the forth. Column 5 could be optional styling information. The user could upload file or paste in the data. When you click on the dot, you get the description. That would be very useful. Users could grab your html+js, hack it up and re-generate an html page with their data hardwired into the HTML/js for sending to collaborators on a science project. This could help people share their, for instance, t-sne plots in a very easy manner.
AFAIK this doesn't support anything but ASCII. Plotting the graphs in WebGL is cool but rendering your own text is arguably an anti-pattern. Even if you think you'd only use numbers you're still likely to run into issues with dates. For example in Japan in many government documents today is 令和３年５月２４日 and not just 2021/5/24. Or if you're showing months in English you can get away with Jan/Feb/Mar/Apr/May etc... but in Japanese you'd need １月/２月/３月 etc... I can certainly imagine in 2021 people would want to use various emoji as labels.
We implemented a WebGL plot + text renderer for https://count.co - adding support for emoji was a pain. In particular you can't use normal SDF maps as for other monochrome characters, so there has to be a per-character flag in the fragment shader to interpret the texture correctly.
This seems really nice in terms of how it's structured, performance capabilities etc., but unfortunately the lack of anti-aliasing (of lines and shapes) and fuzzy-looking text give a first appearance of low quality. (Maybe this just just an issue with how it's rendering on my machine?)
I built CandyGraph a few months ago to scratch an itch and never really got around to talking about it, but it's starting to get some use now, so here it is: https://github.com/wwwtyro/candygraph
I wanted a plotting library that was designed to not only plot a zillion points very fast, but also deal with changes to the presentation of that data at interactive framerates. If I zoomed in on the plot, for example, I wanted the axes to update fast enough that I wouldn't have to worry about the hit. If I changed an axis from linear to log, the rendered data would update immediately; I didn't want the CPU to churn through a million data points and ship them to the GPU again before it rerendered.
So CandyGraph was born. It adopts D3's elegant concept of scales, but it implements them on both the CPU and GPU. When data is rendered, the scaling takes place in the vertex shader. Changing a scale wont cause a missed frame.
On top of scales sits coordinate systems, also implemented for the CPU and GPU. These combine multiple scales into a system capable of dealing with 2D points. It has cartesian and polar coordinate systems, currently.
It renders everything on the GPU, even axes and text. No need to overlay your speedy WebGL plot with an SVG or another canvas for your axis rendering, and no need to take a perf hit for doing so. And of course, the data rendering itself is speedy; it's all instanced by default.
Finally, CandyGraph comes with a small but growing collection of optimized primitives and higher-level objects to simplify common plotting tasks.
If this sounds like it might scratch your itch, check out the tutorial and examples: https://wwwtyro.github.io/candygraph/tutorial/dist/ https://wwwtyro.github.io/candygraph/examples/dist/
CandyGraph is built on top of the embarassingly good regl library. CandyGraph is in major version zero, but it's usable (and in use) today.
[1]https://twitter.com/wwwtyro/status/1396500134716973063