I was floored today when I read your dev entry. Many times you've written things that are exactly the things I think, but your post today almost to a T expressed my life mission. I so like the way you think.

For a couple of years I worked at a company in Bellevue, WA that was (and still is) trying to achieve a mission like this, but it was clear that they were never going to actually release anything so I left and have been working on the idea ever since.

Martin Fowler has given a name to a sort of a form of this idea which is "Language Workbench" http://martinfowler.com/bliki/LanguageWorkbench.html. It doesn't exactly fit the way I think about it, but you may find it interesting.

There's a tool called MPS (http://blog.jetbrains.com/mps/) that shares these ideas that is open source and is produced by JetBrains. Unfortunately it's written and based on Java and has that typical sloppy Java feel to it. The base ideas are pretty great, but it's just so hair-pulling to use. Still it's worth checking out and could be useful for simple tasks.

The code I PMed you recently for the purpose of showing .erb metacoding is the first leg in building a proper structured editor (or projectional graph editor as I refer to it). I always feel the need to clarify that I'm not talking about editing bubbles and arrows, but graph in the CS sense. The graph is projected as structured/formatted text (but isn't text). The code I sent is for a "language" that allows you to write transformations which is how you go from a generic structure (graph) through a series of intermediate "languages" (graphs w/ particular shapes/identities) to pixels on screen while having complete traceability back to the data you're editing which allows you to make it a 2 way projection where you can send editing commands back through the projection. One key idea that I haven't heard you come across yet is the need for strong identity as opposed to names. We conflate names with identity in almost every programming language out there which makes for awful names and weak identities. Every node in the graph should have a GUID. You can attach any number of names you want (different languages, brief or otherwise, iconic or textual, etc) and choose to project them based on the projection you're trying to achieve. Okay well that's a sample of me ranting on the topic.

I'm going to spare the giant rant here, but I have a ton of thoughts/opinions/ideas in this area and would love to talk more if you're interested. I think you would be.

Btw be careful not to get in to watching Jonathan Edward's Subtext videos because you won't get anything on LT done that day: http://www.subtext-lang.org
And definitely do _NOT_ get into Bret Victor's videos: http://vimeo.com/36579366 http://vimeo.com/search?q=Bret+Victor

We have this idea of some logical, structured piece of functionality that we want to create. And then we break it down into this horrendous, text-only language. We then suffer through typos, compiler errors, and all manners of bugs because of the fact that we're trying to communicate this elegant thing in an incongruent medium, and hoping that it comes out elegant on the other side after it's gone through the compiler (which is struggling just to make sense of it all). I'm really starting to think that it's all a bit crazy.

Well, quite.

And if you think that's what you want now, imagine how you'll feel when you've been in the trenches for 30 years fighting poorly-defined requirements, inexplicably arcane high-level language definitions, a profusion of external scripting languages (that somehow must still be integrated and verified), and multiple overly-finicky-yet-inconsistent compilers.

These objections aren't new. I am antique enough to have been around when "computer-aided software engineering" (CASE) became a thing.

The notion behind CASE was to try to cut out the hard and error-prone conversion of functional ideas into ones and zeroes. That is absolutely not a bad idea. But it petered out -- which is probably why very few people on this forum have ever heard of CASE -- because everybody kept running into the same problem: any specification format capable of expressing adequate functional directions to a computer is a programming language... and you're right back where you started.

The point of my saying this is not to pooh-pooh the idea of designing a less broken way of expressing software behaviors. I'm for that. I'm saying it's been tried (and providing a link to a starting point for the literature), and that there's a kind of Gödelian gotcha in that any communication medium powerful enough to express interesting programs becomes a programming language, but that progress in this area would still be awesome.

Maybe we're not going to get to a Star Trek-like natural language "programming" metaphor. But something better than the crazy-inducing patchwork of control syntaxes we've been making do with for over 50 years now would be a beautiful thing.

Go, Josh, go!

Hey negascout, thanks for your thoughts on the subject It's really nice to have similar-minded people around to discuss it with.

I'm very interested in your projectional graph editor. This sounds almost exactly what I'm envisioning! I think the most distilled essence of my position is: we need a better way of editing general structure. Text is not good for that. Imagine a world where everybody had not only their favorite text editor (Vim ) but also their favorite graph editor. A world where every structured data type could be loaded and edited using your favorite graph editor, just as every text file can be loaded in your favorite text editor. A world where we opened and freely edited binary game data in our favorite editor because the developer shipped a "meta-structure description" file that informed our editor of how to parse the structure.

It sounds like this is exactly (or really close to?) the kind of thing you're envisioning. IMO, in that world, we would be years ahead of where we are today. Why do we not have this already?

The beautiful thing about it all is that, just as certain editors define operations and constraints to help with specific data formats (i.e. Vim & Emacs providing lots of functionality that's obviously meant for code editing), we could write plugins for our favorite graph editors that imbue the graphs with structural constraints and perhaps even creation shortcuts specific to the problem domain.

And about weak identity - I agree, I would not want to reference things by name! The only reason we do that is because we are using text and have no better way! With a general graph editor there's obviously no excuse for not having hard references. The ramifications of that in a code editor are just massive. I can see refactoring over there in the corner packing his bags

And by the way, I'm going to reply to your PM sooner or later, I've been having a look at the files you sent me, so thanks

@Flat - I think I understand. So the problem is always we come back to the most general representation - bytes - because it's hard to design another representation that's as expressive. But IMO what we really need is a representation that's even more expressive. I think the "graph" ideas that nega and I are dreaming of are more expressive than text. I mean, they can contain text, but they also naturally express relationships, something that text can't do as naturally. A general representation that is aware of structure is, IMO, the game-changer for programming.

OP, welcome to a concept initially thrown around in the 70's...
4GL

There is other interesting project called HiAsm. Site is in Russian but you can use web pages translation to read it.

If I'm understanding all this... Basically all you guys want is to program a programmer that can then create programs for you?

The way I see it, the biggest hurdle is indeed language. A computer has no understanding. This also why I'm not sure that visual programming is a solution.
We're heading towards a future where the computer is integrated into everything. It will be around us, and we'll probably interact with it through voice most of the time.

I'm envisioning a central computer in our homes that take care of our basic needs. You should be able to program it saying "pressing this button needs to make the light in the hallway on the ground floor go on for 45 seconds". But for this to work the computer needs some sort of awareness. It needs to know which button you're pointing at, and what/where the hallway is and how to send current to a specific light fixture, and how to count time. Arguably we're already doing this, but we're still hardcoding this awareness, making specific programs for each house instead of a toolkit that just works.

If getting a computer to switch on a specific light is already this complex, then having a computer create a game for you is mind boggling complex. I don't think a computer will ever be able to write its own graphics engine for instance. But I do believe that creating environments could be a lot easier. A computer can be programmed to understand the concept of empty space, or a road or buildings. it can download or generate textures. Or even use images from the web to populate an area ingame. It could understand where the camera should be, how high you can jump, if you have weapons or not...
I'm sure you could probably let a computer help you create a game you want to play, but you wouldn't be doing more than offering parameters to a game creation toolkit.

Katorone wrote:If I'm understanding all this... Basically all you guys want is to program a programmer that can then create programs for you?

The way I see it, the biggest hurdle is indeed language. A computer has no understanding. This also why I'm not sure that visual programming is a solution.
We're heading towards a future where the computer is integrated into everything. It will be around us, and we'll probably interact with it through voice most of the time.

I'm envisioning a central computer in our homes that take care of our basic needs. You should be able to program it saying "pressing this button needs to make the light in the hallway on the ground floor go on for 45 seconds". But for this to work the computer needs some sort of awareness. It needs to know which button you're pointing at, and what/where the hallway is and how to send current to a specific light fixture, and how to count time. Arguably we're already doing this, but we're still hardcoding this awareness, making specific programs for each house instead of a toolkit that just works.

If getting a computer to switch on a specific light is already this complex, then having a computer create a game for you is mind boggling complex. I don't think a computer will ever be able to write its own graphics engine for instance. But I do believe that creating environments could be a lot easier. A computer can be programmed to understand the concept of empty space, or a road or buildings. it can download or generate textures. Or even use images from the web to populate an area ingame. It could understand where the camera should be, how high you can jump, if you have weapons or not...
I'm sure you could probably let a computer help you create a game you want to play, but you wouldn't be doing more than offering parameters to a game creation toolkit.

That may be what the >3GL programming languages are talking about, but that's not really what I'm talking about (and I don't think negascout is either).

We're just talking about more powerful constructs for expressing programs. The idea being that text is a very laborious way to specify the structure of a program, because in reality there are so many symmetries and patterns all over the place, and although we try our best to use the constructs available to us in the language, at some point I think the fact that we ultimately must express them in text is a crippling limitation.

Consider this metaphor which I rather like. Imagine an architect. He is told that, with his tools, he can design however big / fancy / sophisticated of a house as his mind can fathom. But, in the end, he will need to express his house - however complex it may be - only in written form, because that's the format that the contractor requires in order to build. So he must develop a thousand little syntaxes and paradigms and tricks for trying to express something so orderly, so structural in a language that is totally unsuitable for it. Yes, he'll eventually get good at it. But the fact that the output medium is so far from what is in his head...it makes him very prone to error. By the time his house has been built by the contractors, he can only hope that he at least got most of the structural translation correct.

That's about as good of a parallel as I can think of, because in many ways, programming is architecture. I mean, when you write it out like that, thinking about trying to express the layout of a building in text form...it sounds ridiculous, right? Absolutely silly. But that's exactly what we're doing, as programmers. We're designing these lovely, beautiful pieces of architecture. But our contractor (MSVC, GCC, Clang) understands text and text only. So we have become masters of describing architecture with words. But there's really only so far that you can go with such a poor medium. Imagine if we were given paper, pens, previz software, etc...

Like trying to describe a picture to a blind person using just words. That's really the whole problem of humanity actually. We have ideas in our mind, pure simple ideas and when we fail to communicate it exactly, conflicts arise.

Im just waiting for brain-computer interfaces atm to circumvent the whole issue xD

So, what you want to do (as an architect) is write or say an instruction, and let the computer extract meaning from that and create the object you have in mind?
But spoken language has the same weaknesses as written language. As a method of conveying ideas and forms it's very inefficient. And yes, this is exactly why we're making drawings.

Perhaps it's also exactly in the architecture world that the computer aiding has been evolving a lot. We got autocad where you could literally write a set of statements and create what you have in mind. No need to even touch the mouse if you don't want to. There are a lot of 3D programs that let you drag/drop and resize structures and combine them into your vision. Some architects even use clay to create their vision and then scan it with a 3D scanner into the digital world.

I have no idea what kind of things you can do with computers. The most complex things I've used are simple loops and conditions.
I guess I do understand that it would be practical to give an instruction "loop through that stack and give me the result that matches these conditions". Where the computer would look at the stack, and create a function that can loop through that kind of information in the most optimal way. But I don't understand where the computer would get its context from. How will it know what the best course of action/method is?

For some reason this topic reminds me of this.
Have some eye and hand tracking instead of all the keyboard tapping, and that's how I'm imagining the future of creating programs.

I always love the "Smalltalk failed so we can't have nice things" argument against building stuff.

Katorone wrote:If I'm understanding all this... Basically all you guys want is to program a programmer that can then create programs for you?

Nope, programmers still express all the intentions, but we do it using tools that suck less.

JoshParnell wrote:Imagine an architect. He is told that, with his tools, he can design however big / fancy / sophisticated of a house as his mind can fathom. But, in the end, he will need to express his house - however complex it may be - only in written form, because that's the format that the contractor requires in order to build.

Couldn't have said it better (and didn't).

Anybody who finds these interesting should check out the google group Augmented Programming: https://groups.google.com/forum/#!forum ... rogramming

I created it awhile back in response to a small but detectable surge in interest in this area that seemed to be kicked off by some of Bret Victor's stuff. There are a bunch of interesting folks there building cool things.

JoshParnell wrote:I'm very interested in your projectional graph editor. This sounds almost exactly what I'm envisioning! I think the most distilled essence of my position is: we need a better way of editing general structure. Text is not good for that. Imagine a world where everybody had not only their favorite text editor (Vim ) but also their favorite graph editor.

_exactly_. We've used the ASCII/UNICODE formats for so long we forget that they aren't special. They _are binary formats_. They just happen to have a wide variety of editors that are good at editing them (I choose my language carefully. There are many great text editors, but why the heck would you want to edit large blocks of text in the first place). We need a format as ubiquitous as text that stores structure and describes ways that the structure can be projected/edited. It will be a more complicated format than ASCII, but this is 2013, we can do it.

Flat is completely right. Previous attempts at such a format have failed miserably for a variety of reasons. Some become too specific to a particular domain and become domain specific languages. Many are based on text and don't do the strong identity thing right, and I'm sure there are other mistakes made.

Here's the structure I propose:
A directed graph with nodes having unique identities and edges labeled by identities. Loops are okay. Multiple edges with the same label are okay.

Simple. Can be stored w/ indexes or w/o. Encoded in XML or json or raw binary. The actual bits on disk aren't interesting to me and I think it's fine if there is more than one encoding for different situations, just so long as the semantics are consistent. We'll never open them in text editors anyway so it doesn't matter except to the guy writing the graph editor. So far nothing shockingly new I think. The key is how we avoid specializing the graph w/ the help of strong identities.

Identities are things that point into some space of things. This is how we get atomic data into the graph. Most attempts make the mistake of picking a few special atomic data types and making them special nodes you can throw in. Identities can be parameterized by GUIDs to point at nodes, integers to point into the space of all integers, strings to point into the space of all string, etc. The idea being that a Node doesn't contain an integer, but a particular node that owns the identity "Int 42" _is_ the number 42. If you point at it from multiple places you're pointing at the _same node_.

Now that we have that layer we can build _on top of it_ (many make the mistake of not layering this and the graph format itself becomes a language) Domain Specific Languages to build ways to give meaning to particular shapes of graphs. There has to be a (at least one, no reason there can't be more) bootstrapping Schema DSL that says "there's a thing called a procedure, it has a name" "there's a thing called a name", etc. There will be some self referencing there. Once we have that we can build other DSLs that can very well be general purpose programming languages. Let's edit C++/Javascript/HTML. No need to reinvent wheels that aren't completely broken and already have a bunch of support. We can even build DSLs that let us describe generators that convert from one graph to another. We can build editor DSLs that describe how to present a graph on screen and make it editable (this acts like a text box, this is selectable, you can delete this, as Josh described we can have all sorts of domain specific commands). We can use the transformation DSLs to transform from input graphs to editor graphs, and boom we have a projectional graph editor. Some of these DSLs can be part of the format spec and be supported by muiltple graph editors which interpret them in their own way. It's likely that if it became popular folks would make better ones and if they're good then good editors will add support for them. Competing standards aren't bad especially when they're built on a common underlying format.

JoshParnell wrote:And by the way, I'm going to reply to your PM sooner or later, I've been having a look at the files you sent me, so thanks

The code is a bit dense and not really intended for public consumption yet and I know you're a busy guy these days so I won't feel bad if you don't really get a chance to dig in . Maybe you want to wait until LT 1.0 to dig into this stuff too hardcore? I'd be happy to write something up to help you navigate it if you want, but for now I'll just suggest you start at Main.h.erb which is the function (and I mean function in the FP sense even though it's c++) that is the equivalent to "int main()" in c++.

Posts merged. The edit button is your friend. -- Grumblesaur

Let me suggest a reframing of the problem, which might help with seeing a better solution than YAPL. (As John Gall put it in the ineffably brilliant Systemantics: "If you have a problem that seems intractable, consider that you may have a metaproblem.")

Back in the dinosaur era of mainframes, commands were expressed as "opcodes": BALR for Branch And Link Register, MUL for multiply, and so on. Naturally this gave rise to lots of joke opcodes.

One of the best-known of these was DWIM: Do What I Mean. I mention it because that seems like what we're really asking for here. We want the computer to Do What I Mean. Rather than learning how to design a house, we want to tell an architect to DWIM. Rather than learn how to build a house, we want to tell a general contractor to DWIM.

There's nothing wrong with that. (Although there's nothing wrong with learning architecture and construction, either.) But here's the rub: the less completely you specify exactly what you really want, the less likely you are to get what you really want.

TanC wrote:We have ideas in our mind, pure simple ideas and when we fail to communicate it exactly, conflicts arise.

Right. You can tell an architect you want a "big, fancy, sophisticated house," but your conception of what those mean may not be the same as -- or even close to -- the architect's interpretation of those high-level requirements. The more you can speak the architect's pattern language, and the more specifically you can use that language yourself to express your goals, the smaller the delta can become between what you really want and what you actually get.

But let's say you still want to use an architect, rather than becoming one yourself. Happily, the real world offers a partial solution.

The problem of asking a computer to DWIM changes when we realize that the computer doesn't have to get everything exactly right the first time. Why expect that? What tool could ever be good enough to read our full intentions directly from our brains? (Would you really want that?)

Maybe what's really needed is iteration.

Directors of science fiction movies don't always know exactly how every single location and prop and costume should look. Instead, they'll take a general idea to their experienced leads, who'll work with their staffs to crank out numerous visual concepts based on the director's general ideas. Then the leads meet with the director and present the proposed sketches to see if any of those ideas satisfy the director's specific desires. If yes, yay, they get implemented; if not, OK, they go back for refinement or replacement. Then those get looped back to the director until they converge on an acceptable solution.

Maybe that process -- iterative refinement though experienced and creative domain experts -- would be the most successful path toward a working DWIM programming tool. Maybe the real solution is a kind of expert system (composed of domain knowledge experts -- plug-ins!) that works with you to iterate through your high-level ideas to reduce them to visual, audio, and textual elements linked dynamically to yield the desired behaviors and outputs.

A visual metaphor for expressing those ideas could be very helpful. I hesitate to conclude that it can be a complete working solution for replacing text-based coding, though. But hey -- you have to start somewhere!

Flatfingers wrote:Let me suggest a reframing of the problem, which might help with seeing a better solution than YAPL.

We're not talking about a language we're talking about an editor and ways of representing languages. I mentioned a few DSLs in my previous post, maybe that's where you got that? The DSLs are just to drive the editor, not to act as general purpose languages. I'm not really interested in a new GPL, just a better way to edit existing ones. We're also not talking at all about DWIM. We'd be writing roughly the same things, just using better editors to do it.

Of course once we remove the shackles of text it's likely that it'll unlock a lot of new ideas that might make the current languages start to look a bit dated, but I can't think quite that far ahead (though some have, my favorite example is Jonathan Edward's various work on Subtext). We almost certainly would at least augment existing languages w/ metaprogramming DSLs, but that would just be C++ macros done better, not some magic DWIM instruction.

Sorry, I should have been more clear that I'm using the word "language" (and "programming language" in particular) in a very broad sense: any syntax for expressing the functional details of an idea.

That encompasses text, graphics, and audio. I'm suggesting that a language alone, regardless of its form, doesn't get at the real problem, which I see as more of a process issue than a tool issue.

I agree with that, flatfingers. Though, I have to point out again that I'm not a programmer and this conversation, for me, is rather philosophic.

In the extreme I'm imagining a holodeck from Star Trek. People create a program by telling the computer what to display, what sources to use, and so on. If you have created the shape of a house, you can then tell the computer to add some decorations. The computer can examine what it has created so far, and from that context decide on accurate/matching decorations.
The computer still doesn't understand beauty though, it just has a huge database of buildings, architecture, people, literature, facts, fiction, ... And it can draw on those to provide it with the needed information to make a good guess. If you're asking for a gothic arch, the computer will probably choose the most popular arch first, and then allow you to refine its selection.

Programming language is still a language. If a picture is worth a thousand words, then you probably do need a lot of words to accurately describe that picture. I don't know about visualizing data or structures. I always imagined that this is done in the head of an experienced programmer. But suppose you follow the analogy. If you want to code using pictures, then you first need to describe each picture in a modular way, so it can be plugged in to other pieces. An imaginary puzzle piece can mean a stack. To this piece you can attach another puzzle piece that loops through the stack, or counts the amount of items, or displays a graph of all the points. But the puzzle piece that displays one item from this stack can't be attached, for that you need the puzzle piece for selection in between first.
Perhaps, like flatfingers says, when there has been enough iterations, people could perhaps agree that these puzzle pieces are "perfect", and work like they should through the context of the other attached pieces.
Perhaps compiling this kind of program would mean that the puzzle pieces write the current higher language for you. Or even that you could create a new puzzle piece out of a combination of others.

For me, the problem isn't really the language, but the computers inability to understand this language, or to get to know you. It took me a few months before I could work closely with my old boss, because his manner of speech and his thinking patterns were different than I was used to. Once we became more attuned, working together became fun and more efficient.