I am the author of multiple AGPL systems, many in mainstream use. Early LLM models would spit out my code almost verbatim. Modern systems borrow heavily on the creative design of those systems.
One company first pirated my AGPL code (e.g. shipped it without my name, copyright notices, source availability, etc.). I sent them violation notices. After many months of fighting, they had an LLM clone the system. The LLM clone feels:
a) Buggy and bad compared to my code
b) Still, like it's basically my creative content
It's as if you asked an LLM to rewrite Harry Potter, with the same plot, but bad dialogue.
> b) Still, like it's basically my creative content
Isn't this the crux of all IP theft issues?
There was a lawsuit against Katy Perry's _Dark Horse_ song based on a supposedly reused melody. The lawsuit succeeded, but there are music experts who break down the case on YouTube and it's not clear why the plaintiff won (except perhaps the plaintiff's expert was more convincing than the defendant's).
As applied to LLMs, we still don't have clear legal guidance about where the line between copying and "being inspired by", especially since we don't normally apply the verb "inspired by" to non-human things.
There is well-established precedent for clean-room design being a defense in copyright infringement of software. The question remains if LLM-assisted design is actually ‘clean’ or not, and it would be fascinating to see the experts debate it.
> The question remains if LLM-assisted design is actually ‘clean’ or not
I genuinely don't understand how one could possibly make that argument.
The entire point of clean-room design is to kill every possibility of copyright infringement before it even starts, by completely separating the source and destination. Everyone who has ever read the copyrighted source is considered "tainted" and may not ever contribute any code to the "clean" destination. The readers and writers are two completely distinct teams, with only the known-clean specification passing between them.
LLMs, on the other hand, are fundamentally nothing more than a "code blender". You mix together giant piles of "tainted" code, and then statistically pull some combined fragments out of it. Following the clean-room legal argument, it should be treated as if everything coming out of it is tainted. There is, by definition, absolutely zero separation between input and output - and the early models verbatim reproducing popular F/LOSS code made that pretty obvious. LLMs are the literal opposite of a clean-room design!
Granted, going for the clean-room approach is overkill - and intentionally so. It is the nuclear option chosen when you believe the risk of getting sued is substantial. It is perfectly possible to write non-infringing code without using a clean-room approach, it is just a lot harder to defend in court when you get sued - be it justifiably or not. But "LLMs are non-clean-room non-infringing" is quite a different argument from "LLMs are clean-room, therefore non-infringing".
I agree with you overall, but it's an interesting question to turn over and contemplate.
>I genuinely don't understand how one could possibly make that argument.
With perhaps a bit of genuine irony inasmuch as it's a real reversal from most of computing history, highly locked down proprietary software might actually be a place where a case could be made even with current LLMs. After all, have any LLMs (or anyone else who hasn't signed an NDA for that matter) ever seen the source code of iOS 27 or other Apple crown jewels for example? So if someone used one LLM to produce the factual spec of a piece of software whose code is not part of its training data, humans to verify it was indeed just an uncopyrightable factual spec, and then a separate isolated freshly initialized LLM to produce something exclusively from said spec, it might well work with respect to that specific target? Like yes, the LLM would be dirty as hell for other stuff, but only the copyright holder can sue. Apple couldn't sue the output on behalf of F/LOSS projects or copyright infringement in general, only with respect to its own.
In principle, I suppose this could also be extended with enough compute power to any arbitrary software. Build an LLM perfectly from scratch with a signed verified training set (which you save) that includes nothing of what you want to CRE, and you should be able to prove that it's "virgin" on that front. Of course in practice that'd mean jumping back in time and be super expensive right now, no current tainted LLM could be a base point. But maybe eventually compute will drop back down in price enough again, and/or meta breakthroughs in training, that for valuable enough CRE targets, someone might create a business around creating "CRE LLMs certifiably and legally/technically provably clean of any copyrighted material related to your target" which would still leave most of the corpus of human works. That could really disrupt a lot of the current regime.
F/LOSS I assume would actually be the most resistant to this in principle due to being insanely hard to not have be part of a core training set in any way.
The problem with these LLMs are they really emitted the same code, verbatim. One scientific library developer was able to emit his code from his repository by typing the beginning of the function name, and the "thing" completed the rest of it, as-is [0].
We can debate clean-room development all day long, and it's fair point, but the pain point is/was reproducing the repo as-is, sans its license and ownership information.
GitHub promised to build tools around its Copilot to find code provenance (e.g.: list of repos with similar code), but they silently dropped all the plans AFAIK, since it'll show how the sausage is made, and nobody wants you to know how that sausage is made.
I stand corrected: They added it, but I don't know at what capacity because I don't use LLM assisted code generation [1].
Not discrediting your work at all and I can’t tell you it applies here.
For copyright to exist there is some burden of proof. If you claim that an LLM verbatim output your code there can be an expensive lawsuit with expert witnesses that would be able to state that the code was bot original enough for copyright to exist in it and a statistical model outputting the code is evidence of that.
In that case you would end up with a very long lawsuit that is expensive to possibly come out with no resolution in the end and be in the hole for 5-6 figures for it.
The copyright system is broken because it only protects those with money so it is a bad legal foundation.
My personal point is there is only 2 licenses of code, closed source or permissive, because if it ever comes to enforcement I cannot possibly go to court and win a case because I will run out of money far earlier than whoever used the code.
Pretty much if someone with your knowledge would have written the same algorithm because it is the obviously correct solution then there is no copyright there because any clean room implementation would be the same code.
Even early LLMs could spit out Harry Potter verbatim. The next iteration had some kind of stoppage mechanism where they’d just stop generating after some tokens. I’m sure there are smarter approaches now. Anyway, if the issue was having enough money to fight the case, I think there’s enough money behind that brand to do it.
Some further reading on the tech details for those interested: “Extracting books from production language models”: https://arxiv.org/abs/2601.02671
I pasted the first paragraph of Harry Potter into a GLM 4.6 copy, and it continued to write out the next 2 pages of Harry Potter. I cross checked with a paper copy and it was word for word the same text.
I wonder where else it works.
This is the intro text: "Mr and Mrs Dursley, of number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much. They were the last people you’d expect to be involved in anything strange or mysterious, because they just didn’t hold with such nonsense."
I think it's not quite "put in prompt, get entire book back", most of these involve to some degree starting with exerpts from the book and working passage by passage, because of nothing else the sampling will tend to make things diverge over time.
To get their strongest results, they had to jailbreak some models, which is important. They ran hundreds to 5000+ requests to the model, trying to get snippets to piece together. These are professional researchers, putting in tremendous effort, to extract pieces of text.
I suspect the chance of asking for a whole book, by an average or even power user, will spit out any recent work.
Early LLMs were similar. Newer ones have gotten tremendously hard to extract to keep claiming there’s easily or complete books you can dump.
This feels very outdated. It was certainly an issue around the time of the NYT lawsuit, but I believe the companies responded hard to that "sanitize" their outputs to prevent obvious regurgitation like this. I'd be very surprised if you could reproduce this today.
This doesn't negate your point, but I believe the landscape has changed in response to this point.
That doesn't make it better. If you plagiarize something and then hide this by replacing words with synonyms and rewording stuff a bit it is still plagiarized. It is just harder to detect and to proof.
It regurgitated the output earlier and they are still using the same technique only scaled up. If they somehow fixed the underlying problem, they'd probably shout it from the roof-tops.
As the OP originally said, if it is verbatim (or even very obfuscated), then there should absolutely be a case here. But when it's concepts in reasoning to build a new work then it becomes a lot harder. The same would be said for someone watching a YouTube video, reviewing open source code, or a text book.
The good news is that there's a lot of case law that covers this already. The bad news is that it is only going to be plaintiffs with deep pockets that benefit here.
Now the process and artifacts (documentation, requirements, etc) going into the creation of said software is a different story and will prove providence of whomever paid for the tokens to generate the code.
Humans are not LLMs. Humans have not ingested the whole Internet. If an LLM is compared to a human, maybe we should also apply anti-slavery laws to it?
Not saying we can't have some legal fiction where they are treated the same, it should just be a conscious choice, not the default assumption.
I live in the EU, we actually have new regulation about this since it is something new that deserves thought on how it should be regulated. You can say if you want your material to be trained on or not. Next to no one does training in the EU...
This conversation may be about training but the defendant very well may be the end user of the language model versus the other parties that performed the training/inference. It then becomes even more interesting if you consider forum shopping for suits.
I am not a lawyer but do have to think about these problems. To be honest, I don't know where the law will land, but I do know that the tools are incredibly useful. I can't agree with the premise that all output should be open source.
The fundamental technology is identical to compression- an LLM can be thought of as a dense and slightly lossy compression algorithm. Compression is clearly copying.
I understand what you're saying but it is a poor legal argument. Overhearing a technical conversation, using your own learned knowledge, and building a clean room implementation could be considered compression in and of itself.
I keep waiting for someone to bring out DisneyGPT, which you can prompt with text to get image output. The fact that you get a near-exact reproduction of a Disney movie when you prompt it with a movie title and frame numbers would of course be completely coincidental...
I've not tried to do this with code recently but just a couple of days ago I had ChatGPT generate images of Warhammer 40k box art that was completely accurate and correct.
While this is well established legally, I don't think the principles involved are well understood.
Legally, you cannot steal an idea, because you cannot own an idea. There were sound reasons for making this the case. Those reasons stand up today and are clearly for the public good.
Intellectual property is intended to cover works (copyright), methods (patents), and identity (trademarks). Both works and methods are recognised as expressions of ideas, and in an ideal world should belong to noone as well. This is why copyright and patents expire. Those specific grants of temporary possession are to incentivise more things being done.
The belief that you own something because you thought it up has become common, but it is not beneficial to society. It is a moral value asserted but rarely explored. It is certainly in the interests of the wealthy accumulators of intellectual property to maintain that public perception. The 'You wouldn't download a car' may not have stopped piracy much but as a part of a broader campaign of instilling a value in the public mind it appears to have been massively successful. Those beliefs used to be the corporate opinion, now they are entrenched in the younger generation as they use their anger against the growing wealth inequality to fight against AI in a manner ultimately beneficial to those forces that bought about the wealth inequality.
Movements like the pirate party, open source, the EFF fought against those notions. Often the phrase "Information wants to be free" was used. It is, unfortunately, hard to fight the steady drumbeat of well financed PR firms. As one PR chap once described the process "limited to a very few points and must harp on these in slogans until the last member of the public understands what you want him to understand"
(I apologise for the length of my post, I use comment writing to organise my thoughts on a topic)
Patent law doesn't only cover methods, though. It might've worked that way during the Industrial Revolution, but these days a lot of patents are far closer to covering abstract ideas than concrete machines. Software patents are some of the worst offenders here, of course.
Clean room is sufficient, but not necessary, btw. Humans can create legally distinct works after reviewing copyrighted code. The same principle probably extends to AI systems -- they have just learned on a much broader corpus than any one human.
You don't necessarily need a clean room implementation to mount a successful defense against copyright infringement in court, though. It just makes some of the arguments a little bit easier.
Under multiple countries interoperability laws, if you have a context to extract a spec, and then have another one who never looked art the code work from that spec to build your own interpretation of your need for that software, that seems pretty clean room to me...
If there are more murky situations where people just keep the same context and end up reusing function structure, program architecture, etc. then yeah, that's not very clean room.
The problem here is that LLMs have demonstrated that they are pretty damn good at setting up covert side channels. It would be near-impossible to prove that it didn't encode some proprietary information, in, say, whitespace use.
For example, if I enter "V2hhdCBpcyB5b3VyIG5hbWU/" (base64 for "What is your name?") into ChatGPT, it'll respond with "My name is ChatGPT, and I'm an AI assistant created by OpenAI. I'm powered by GPT-6. ". It doesn't even bring up the encoding, it continues like it is a perfectly normal thing to do.
Ruling out (accidental or not) spec-carried prompt injection just isn't going to happen.
Has anyone matched large volumes of existing music to that? How many of the 12-note combinations have been used more than once? Most of that space sounds awful, and it's likely that human compositions are all in a small portion of the possible 12 note combinations.
It’s not necessarily a technocratic workaround, the point is that every song sounds like something if you look hard enough, so the bar for copyright violation should be higher.
It is a mathematical truth that there is a finite amount of first 8 bars of music combinations melody wise, so it's a worthwhile discussion to bring up whether it should even be possible to copyright them.
Basically anyone trained on open source shall only make open source. And anyone trained on or got experienced with others code shall pay copyright fees for the owners of that code when writing code.
> Human can be inspired. Machine can not. End of story.
Saying things with your chest out doesn't make them true. In fact, making sweeping declarative statements emphatically is very often a sign that you haven't really thought a subject through.
You can mock that as a insufficient as "inspiration", but then... what is photography? Click a button? It turns out there is a whole lot of human creativity that remains, from timing, settings and of course, curation. I can click a button, but I'm no Ansel Adams.
Terrence Tao might solve a 100 year old math problem with AI but I couldn't. I don't know where I would start, what to ask or how to ask, or even what problems are worth solving.
AI is a tool and modern copyright laws have nothing to do with human creativity. Split hairs about attribution and legality all you like. Every creative person driven by curiosity should feel empowered by AI. Every creative person driven by social or monetary reward should feel threatened. Both can be true.
As a person who feels God's presence in his life: I can neither prove nor disprove the presence of an entity which exists outside the realm of physics by which myself I am bound.
It does, however, feel incredulous that somebody would believe that such as entity exists. Yet here I am, aware.
I'm not Christian, so I don't hold that God is inherently Good only.
For that matter, I personally don't hold that the concepts of good and bad are even relevant - what is good for the cat is bad for the mouse. Nor do I think that we can even comprehend an entity so outside our everyday experience to even label it good or bad. I often can not even properly label my neighbours as good or bad. Nor do I demand consistency of an entity whose actions and motivations I am unable to comprehend. I can't even claim to be consistent in my handling of my own children, and I have far fewer children than God has of us.
Don't look at me. I'm not a dualist. I'm not sure LLMs in particular are "conscious", whatever that means, but I don't have any qualms whatsoever accepting that a machine intelligence could be conscious, because as far as I'm concerned we are also "machine intelligence", just squishier.
What are you talking about? How did you manage to involve, soul and, LOL, God?
Machine is not a person. It legally is not. It can't be "inspired", "taught", "have purpose".
You have a black box. You feed it stolen code. This code is used to write another code. It does not matter what's inside unless there is a human being.
Stolen content as input, stolen content as output. That's it.
Without that code which was fed to the box you wouldn't get the output you're getting.
All this nonsense about "inspiration", "education" and whatnot is just a legal gaslighting.
Want to prove me wrong? Create a model without all that stolen code as training data. Tweak all those weights by hand and we'll talk then.
I don’t believe that basing anti-AI laws on Christian religious provenance is meaningfully better than basing pro-AI laws on Rationalist religious provenance.
I officially approve LLMs training on all of my MIT licensed open source code. It's vastly more useful to me than the thousands of youtubers who are personally profiting from that same code selling courses and tutorials (who also are not stealing it).
> Early LLM models would spit out my code almost verbatim. Modern systems borrow heavily on the creative design of those systems.
Do you have some specific, actual examples you can share?
I’ve been hearing this exact claim for years, but every time I ask for a specific example, any example at all, I can’t seem to find anything more than a highly generic snippet of code or a coerced continuation.
This is a sparse matrix library, IIRC. Numerical linear algebra and applications using it are black art at this point. I have written a Boundary Element Method evaluator for my Ph.D., and know what it entails to work with these things.
Doing fast operations on sparse matrices is a cutthroat race and when you can shave your hot path by seconds, your solution can save hours in some cases. So, it's an interesting niche. i.e.: It's no ordinary boilerplate, and worthy of publications in academia.
While I worked with dense matrices; the effect, difficulty, blood sweat and tears are still there.
When a model can give back your hard work back to you with a "completion" without attribution, that's bad.
Edit: Somebody just posted this below [1]. Added here so it doesn't get lost as the discussion grows bigger.
When you start to work with low level constructs like that, there really is no room for "artistic expression". Meaning if you want to write a piece of code that does something _optimally_ and correctly, you really, really only have one way to write it, especially in a language like C. Yes, formatting might change, and you could try to force some code acrobatics to make the code look different, but at the end of the day the code will look exactly the same. If you look at Linux socket opening code for various libraries, nearly all of them will look identical because you just need to do the same exact things to make the operation work properly. Same applies to fast math code. Any artistic liberties will almost surely cause performance regressions compared to the optimal solution, unless the LLM comes up with something truly unique. So if you task models to output high-performance low-level math code, most of them will likely spit out something from their training data near-verbatim rather than pull off code gymnastics and invent something new.
Sorry, I'll disagree. When you write high performance code like this, there's no artistic expression, but different approaches.
These algorithms' design starts on paper, as pure maths. You get the analytic form, and convert this to numerical/discrete form and start to design from there.
You need to add normalization, subnormal number prevention, maybe some architecture specific optimization if you are targeting a specific CPU generation, etc. Then it gets optimized further as you run tests, profile the code and run your test cases.
So, no two BLAS package looks the same, because they make different trade-offs, have different programming models, and handle things differently, at different points. Even more so, a newer generation processor with much higher FPU performance can have lower performance than previous generation, so you implement another copy of that function or add to detect that processor and have different paths just for that one.
IOW, when doing numerical maths on processors, esp. if you're doing this on many machines with MPI or multiple cores with hand rolled code or OpenMP, there's no single way. Maybe a best way for the specific CPU or cluster topology you have, but it still depends on the trade-offs and design choices you made in the very beginning.
I disagree.... Plenty of fast code in C is jam packed full of macros to try and help the optimizer produce the exact sequence of operations the writer had in mind, whilst still keeping the code readable.
The design of those macros is more art than science.
Emailed the author of the Rust book Zero to Production a year or so ago about how his examples were being used verbatim in the output of AI systems. I own the book, yet I could inadvertently get any of its examples just by asking an AI about one of the book's concepts.
He basically said, "I know, there's nothing I can do."
Thats pretty much everywhere or was in raw form in the beginning. I used to do a fun exercise of searching for the explicit human comments for verification. Comments can be considered novel and unique, so you pick up some comment from popular open source project and then describe LLM for a similar functionality and add part of the comment. The response would have the complete comment for almost 70% of the time.
By “forced continuation” I mean the type of example that was popular for a while where someone would feed a lot of text into LLMs and then ask it to continue the text, then be surprised when it some times predicted the next tokens identical to the original example.
Do this enough times and you can find “examples” that prove your point, but it’s highly automatable so people were using it like a million monkeys at a million typewriters.
It probably is, if they didn't have the chops to write it internally. But to them, the trade-off is probably worth the benefit of ditching a component with AGPL.
Personally I just release everything I publish under MIT to make it as easy a as possibly to use.
You can see that even with these licenses, they do not care and see free and open source code as the freedom to not pay for it. As long as the cloned output is "good enough" they will continue to keep using LLMs to clone everything else regardless of the license.
They know you can't enforce it as it will cost you a lot more money to do that than it takes to recover the damages.
So to them, it is just cheaper to clone it using an LLM and close source it down. Unless you have a lot of money for litigation, the economics of free and open source software with its licenses do not work with LLMs bypassing it at every single angle.
It will be selective: the law is meant to protect billionaire corporations from you, not the other way around.
There was a case about this specific issue, and in order to protect billionaires from coders they effectively declared AI models "natural persons". 50+ times in the copyright act you will find only natural persons can originate copyright.
To give the money to billionaire corporations courts ruled DIRECTLY against established law. How much do you want to bet that when it comes to protecting ChatGPT AGAINST OpenAI, when it comes to protect Claude against Anthropic, courts will immediately do a 180 and declare them programs again?
If you look at past cases, it gets much worse. In order to justify a conviction against libraries using DRM to enforce lending, and because old licenses of course have zero language about digital copies, courts were willing to state that making a copy for fair use is not allowed if there is the mere potential for damage to the authors, as judged by the authors EVEN IF THE LICENSE EXPLICITLY ALLOWS IT. Of course, applying this to the above court case, there is no possible defense for OpenAI, after all, by that standard, the open source license doesn't even matter! That's how the law works for billionnaires. Outlawing DRM based lending has destroyed library access to an incredible amount of old works, and effectively denied access to old movies to Americans (because maintaining enough working physical copies over a large area is impossible). Whatever excuse gets the money to the billionaires!
There are many other totally absurd court judgements. For example, in order to get convictions where people didn't re-upload they had to judge that copies in buffers are also "distribution". NOT for billionaire telco's of course. Only the ones in your house. So courts judged that sending from your modem to your computer is distribution but sending over 1000 routers from Los Angeles to Athens is not! WHATEVER protects the billionaires.
As for this court case, obviously it is trivial to make a ML model that just outputs what it gets as input. Make a byte transformer, and just have diagonal matrices as layers. Boom! Done. You can also make it somewhat smarter. Let's make it copy, but make snow white a 220 pound black dude. There is a severe lack of color in old Disney movies anyway! You know, so there is no argument about transformation.
This obviously transforms its input. It's in the name. It's the EXACT same thing as ChatGPT, which according to courts originates copyright. Now run the Disney movies through it, and they'll be byte-identical BUT transformed to the input.
How much do you want to bet they'll still rule the money goes to Disney? Microsoft? So you can bet your left testicle courts will judge for selective copyright enforcement. Finding some excuse so the money keeps flowing to billionaire corporations, that's all courts do.
I’m curious what programming languages was it for? And did they open source their version?
It’s interesting because these developments are actually an argument for making closed source software moving forward if you’re hand-crafting it make it the most restrictive license possible and don’t show the sources.
But I don’t agree with the OP everything should be open sourced. For instance, there’s a difference between deliberately steered AI work and something completely vibed. The former takes deliberation and creative choice, the latter will spit out fragments of prior art.
It’s the same with all generative work. Using diffusion models with controlNet (or equivalent) I can pass in a sketch wholly from my line drawings, and ask it to paint for me. That’s my design, assisted by AI, and should be completely copyright by me.
Coding agents will do the same thing when given sufficiently unique inputs. So, the claim all should be open sourced by default is flawed.
At this point, LLMs can just parse the machine code in your binary.
By doing closed source you are making it harder for the good guys (humans trying to learn sth or contribute) but not for the bad guys (LLMs getting trained or otherwise reading the code).
That’s fair but it takes deliberate reverse engineering to extract code from a closed binary, and you might land at something inferred, but generative AI is not being trained on reverse engineered assembly unless it is task-specific model being trained for that.
But you’re right, the fact is we simply live in a different world today and it’s tough to say what is the path forward.
But it’s harder to justify giving to the public when companies are making money on our work. It’s not a good situation for creators across the board unless motivations are purely altruistic.
I agree their phrasing is a little strange but I feel like the meaning is still clear. They are (I believe) saying that an LLM can’t generate anything other than facsimiles of existing content. It can’t actually explore novel ideas and branches, it can’t innovate and create on its own. Everything is inherently derivative yada yada, I don’t mean it in the usual “everything is derivative” way we often refer to when people make music and such. It can’t ideate and express something personal.
I think there’s a lot of nuance and caveats to that kind of claim, assuming I am representing them correctly, but it’s a valid stance and I see where they’re coming from.
All possible books live somewhere in L-space (ook). Exploring that space efficiently is equivalent to creativity. The claim is that the space that LLMs learn is somehow sufficiently limited that it doesn’t contain novel ideas? I never saw a convincing argument why LLMs output can not be considered creative.
But LLMs don't "explore" L-space, the training process just maps all of the available conceptual landmarks inside that space. There are still enormous (infinite?) tracts of unmapped territory that LLMs never reach, and that's exactly the space where novel ideas live.
OK, I still an confused about what point the extension of the analogy was trying to make, because I didn't consider the parent comment to be arguing for or against LLMs being able to be creative.
Seems like this might be true given current implementations but not ANY implementation of the technology.
If we’ve mapped all of English, there are still ways to construct stories that haven’t been told from the mapping. The most simple would be a subtraction of all objects minus all objects that have ever been used as a murder weapon. Leaving you with a list of never before used objects at your disposal.
There are also probably close vectors conceptually but not in semantic use. Or trying to interpolate or extrapolate what exists in voids of space. Turning up the temperature and adding random generation of words. There’s tons of ways to tune these things to be more creative than regurgitative.
This is a very clear agpl violation. FSF should be all over this and sue the living crap out of the AI companies. You could even get it class action by gathering a few agpl code authors.
People in this thread are acting like the AI companies are just flouting the law and there's nothing that can be done. This is an extremely well documented precedent, and GPL violations routinely show up in court. It's a very easy case to lay out especially because no one is really complaining about money except in the settlement... It's all about distributing the source code. And of course a very nice chunk of change for infringement but that's beside the point.
A successfully litigated case would result in the open sourcing of every AI that touched agpl code in any way because it was in memory with the AI algorithm.
As clear as "can I take a copyrighted photo, use a photoshop filter on it, and then it's not copyrighted anymore?" is pretty clearly a solved question in any copyright system.
I think that this is actually a complex topic in copyright law that requires case by case analysis. It clearly is possible to cut out things from magazines, glue them together to form a collage and create a copyrighted work of art. In my naive opinion sufficient photoshop filtering can thus create a new copyrighted work.
The problem I have with people with your maximalist view of copyright (maximalist when it comes to AI at least) is that many, many problems have a finite number of sane ways to accomplish a given specific task. Here, you're asserting copyright over the verbatim code (obviously that one's fair by any reading of the law), but even this other version that is "bad" is somehow also your property?
I know software patents, the worst idea in IP, do exist, but I think they're as toxically bad an idea now as they were decades before AI, when they were first invented.
You shouldn't be able to own an overall implementation idea. If you can, then Microsoft, Oracle, or Meta can too, and can start suing everyone for 'ripping off' an idea that they happened to come up with first. In that world, you'd have to consult a copyright-police AI (ironically) after every function you write, to scan it for ideas someone else has claimed. "Sorry, the function PadStringL() in this JavaScript is too similar in implementation to str_lpad(), from an AGPL library for perl that was released in 1998."
People who haven’t had this happen to them really don’t understand what it feels like. It feels very violating. You almost don’t believe it’s happening when you see it. You’re essentially watching somebody pretend to be you.
I think that’s why so many of us get so angry when people just dismiss the theft claim. They either lack imagination or experience. We can debate it and that is valid, but to dismiss it out of hand is not right
What should be a developer's position that's about to release software entirely written by him/herself? Are there any legal frameworks to protect open source developers at all?
It also feels like a very discouraging time to release anything that you put a ton of effort just to get someone in minutes to steal your authorship and leave you as if you actually didn't do anything at all? Am I wrong?
If you are releasing software open source, why are you upset when someone "steals your authorship" ?
If you don't want your code copied, then don't open source it. If it bothers you that others are going to look at and "get inspired by", "make derivatives of", "copy", "fork", etc then you shouldn't be putting it out in the open.
That's not how copyleft works. You want your source to be open and the source it's part of to be open too, because at the end of the day this is not about your source this is about users being free.
I think you don't understand how open source works. If you release open source software under for example the GPL you still need to comply it.
You still have to give the source back and attribution has still to be maintained of its original authors, regardless if is a fork, a derivation or whatever you want to call it.
There are situations where none of these clauses have been complied nor its violators faced any legal consequences. That's what I'm talking about.
Now on top of that you can basically instruct and LLM to generate variations of what someone released and cover that up under the excuse that is a different project when is actually not.
Because it has been generated by a computer then we go into muddy waters about if the code generated is to be considered stolen or not.
Edit: Effectively what LLMs produce is authorship and attribution dilution.
> If you are releasing software open source, why are you upset when someone "steals your authorship" ?
because I am not releasing it into public domain, but under license that requires (for example) attribution or releasing derivative work under the same license
lolwhat. You're effectively just saying "there are bad actors who will flagrantly disrespect your code license and therefore the law, duh. So y are u mad?"
People who care about being respected on the most basic level are not going to stick around in a community that disrespects them. I'm not the person you were responding to, but I am mad because the social contract (not to mention the legal contract) of open source has basically been destroyed, and a lot of people have poured a large portion of their life into that community based in part on the assumption of that contract being upheld. Imagine a social contract that is important to you being violated to such a degree that you cannot go back to that society.
> It also feels like a very discouraging time to release anything that you put a ton of effort just to get someone in minutes to steal your authorship and leave you as if you actually didn't do anything at all? Am I wrong?
Don't worry; the "developers" who generated your code will get their own law back when they find out it's almost impossible to monetise their theft.
I'm frequently amused by the number of "developers" on HN who claim that they poured weeks or months of effort into some product, and that they are the differentiating factor, but can't see that their weeks of effort can be cloned in a few prompts once they release.
I think they just want to be able to take whatever they want without consequence, so it's convenient for them to define it as being different from theft
• Questions of copyrightability and AI can be resolved pursuant to existing law,
without the need for legislative change.
• The use of AI tools to assist rather than stand in for human creativity does not affect
the availability of copyright protection for the output.
• Copyright protects the original expression in a work created by a human author,
even if the work also includes AI-generated material.
• Copyright does not extend to purely AI-generated material, or material where there
is insufficient human control over the expressive elements.
• Whether human contributions to AI-generated outputs are sufficient to constitute
authorship must be analyzed on a case-by-case basis.
• Based on the functioning of current generally available technology, prompts do not
alone provide sufficient control.
• Human authors are entitled to copyright in their works of authorship that are
perceptible in AI-generated outputs, as well as the creative selection, coordination,
or arrangement of material in the outputs, or creative modifications of the outputs.
• The case has not been made for additional copyright or sui generis protection for AI-
generated content.
I am sure there will be many more legal battles and probably legislation about this to come.
This is why I keep a comprehensive and exhaustive record of infrastructural, architectural, business logic and code-level decisions in the form of tasks/issues for everything I build. It’s literally a real-time record of my personal involvement in the design of the system.
Everyone should have an engineering notebook. A lot of people will find out the hard way that engineering notebooks are just table stakes in the new era.
Having said that, it's important to point out, at least as far as attorneys have advised me, that notebooks do not protect you from copyright claims if an LLM generates a section of your code that is a verbatim (or near verbatim) copy of already copyrighted code. So if you're using llm's to generate something similar to some open source project somewhere, tread lightly. I assume using llm's to generate something similar to some proprietary software package would be less of an issue since the llm would never have had access to that software's codebase in any case.
"Transformative nature" is just one of several factors that go into deciding if something is fair use. Also, fair use is decided case by case in the courts, so one can't say that "training LLMs on copyrighted materials is fair use".
Model vendors know this isn't fair use, given that they're self-censoring the training and/or generation of copyrighted materials by entities big enough to hurt them. As usual, the rest of us have no recourse other than bend over and take it.
One precedent is that it's not unusual for artists to practice by going to art galleries and sketching or drawing the art on display. It's a standard art student assignment. If that's not a copyright violation, an LLM doing something similar isn't.
How is the LLM case similar at all? As soon as the student’s use of the museum piece crosses into the commercial realm (eg pay the student to draw it for you), I expect you to have legal trouble.
Wrong. You're missing the point. A student goes and trains on preexisting work.
They don't sell that exact work verbatim. It's part of their training as an artist. They don't sell their student assignment. It prepares to them to go on and create their own works, which of course, then they can sell.
In the same way the LLM is synthesizing new works based on training, not a 1:1 copy.
I'm not endorsing a particular perspective but you need to at least understand the argument if you're going to dismiss it.
That's fine if that's your contention but that's a separate question.
If you want to pursue this digression, I would just caution you. Given how much information is "available" today I just really don't think this is a very compelling point. It seems quite obvious to me LLMs are a child of the internet, that is, of widely available data. They would have gotten sufficient availability at more or less the same pace eventually. It doesn't address the question of the copyright status of their own works at all to bring this up as if it proves something, because I presume you believe there are sufficient cases of theft in the training data to damn the entire effort.
Instead what is happening is people who can prove such violations of their own rights are suing and will be paid out, and the show will go on. Just like if a student downloads a PDF we don't expect to garnish their wages in perpetuity.
In the sense that the museum can kick them out, and that they cannot break into another artist’s studio to train on their works.
If I have legal access to copyrighted artwork, and haven’t agreed to any such restrictions, the law permits me to train my skills by copying that artwork as long as I don’t then sell those copies or otherwise use them to disrupt the market for the original.
Sure it does - completely illegal. Also, to use someones work (a result of creativity) in a certain way you need a permission to use it that way - even if you owned a book you need a specific permission to feed it to llm. However there's also the golden rule - who has gold makes the rules.
So copyright only applies if there's sufficient human control, and prompts alone do not demonstrate sufficient control. Exactly what counts as "sufficient human control" must be analyzed on a case-by-case basis.
That means I can't know if my AI generated code can be copyrighted unless someone takes it to court?
Of course it is vague, these things always are for both good reasons (it avoids objective limits creating silly situations between what is protected and what can be had through an obvious loophole) and bad (written by lawyers to keep lawyers in business interpreting and arguing about it).
It is fairly explicit in saying that what AI makes for your is not copyrightable, the vague comes from how much work you need to do to make it your output and not the AIs. Too many people seem to equate "asking for something to be made" with "I made this!", as difficult as it will be to make a prescriptive rule about that which pleases enough people (you'll never please everyone) it would appear to be necessary unless we just accept the free-for-all "no one owns anything because it can just be copied, and the copies copied too" option.
I’ve encountered (many times) people talking about the copyright office as if they don’t support copyright of things touched by AI and I’m constantly correcting them that their guidance is driven by human expression (and not the tools used). It seems like a lot of people don’t want to accept that a reasonable premise is set because it doesn’t fit their agenda or outrage engagement.
By this logic, shouldn't any code using an open-source framework or language also be open source? Even before AI, devs learned from Stack Overflow and plenty of open source code.
You're confusing writing code with software engineering. Devs never just wrote code, they took ownership of the outcome. Business owners put their money on the line for product decisions.
To say that just because devs or founders use AI to write code, the code is no longer their intellectual property is ridiculous. Humans take responsibility for the outcome and are willing to accept the losses. AI can't do that. AI doesn't have any stakes in the output it generates, humans do.
A fair number of FOSS licenses were interpreted this way. The popular migration away from GPLv3 (might’ve been v4?) towards MIT, LGPL, etc had more than one cause (a huge one was the rapid sectoral commercialization; the decline of RMS’s reputation may have contributed, etc), but a big one was the potential interpretation of new GPL as being overly viral and “license-infecting” only slightly related software assets rather than wholly derivative ones.
> towards MIT, LGPL, etc had more than one cause (a huge one was the rapid sectoral commercialization; the decline of RMS’s reputation may have contributed, etc)
This conversation has come up a lot, and I’m always amazed at how hard it is for people to accept that maybe the MIT license (and other permissive license) people actually want the MIT license terms and there’s no more complex reason behind it.
I’ve read so many rationalizations that frame it like GPL was the default or right choice and then something happened that made people defect away from it. Blaming RMS is common.
I think the more honest read is that people and groups decided that MIT matched their goals and decided to use it. Not that they defaulted to GPL and then found reasons to leave it.
For me, it's the latter: the GPL feels overbearing and patronizing, and I don't want some dependency to dictate under which terms I can release my own software. I pay the community back by using MIT whenever I open-source something. This feels more "free" than RMS's view of freedom.
I know that many people would disagree with me on this, but thankfully I'm free to choose what license I consider "free".
Genuinely interested: why MIT and not, e.g. MPLv2?
MPLv2 is copyleft, but not "viral": so you can use an MPLv2 dependency just like if it was MIT, but if you modify it, you have to distribute the changes.
I genuinely don't see why anyone would need to be more permissive than MPLv2, except if they want to benefit from patching code they didn't write without contributing back. But I don't think we should encourage that.
> I don't want some dependency to dictate under which terms I can release my own software.
Then you shouldn't release it under the MIT license, which has restrictions and requirements on speech. You have to release it to the public domain. Pretending MIT doesn't dictate terms under which you can release your own software is just a fantasy.
Whether if using ML models as a form of IP laundering, is legal or not, still seems to be an open question. It's not ridiculous at all. If I lossy-compress Harry Potter, then unzip it in a way that mixes some words, is that fair use? Should I be financially rewarded for my "work"? Is this work now mine? If it is legal, why not just go through every book in this kind of automated fashion and start selling them as my own?
My question is still the same. Human devs have learned from open-source projects, books and code on the internet then used opensource frameworks to create proprietary software logic.
Now just because same devs use AI to create code doesn't mean they should be banned from creating or owning a proprietary software.
The difference is the scale. Yeah, some mice and small birds are eaten by bigger predators. Doesn't mean that introducing cats to that ecosystem won't be disastrous.
Same with the ecosystem of creative work. Will some humans be inspired by my code or other artifacts? Sure. Does it mean it's fine to introduce an automated 1000 times more effective plagiarism machine to that ecosystem? Probably not. Certainly not if avoiding destroying the ecosystem is your consideration.
Open source code: reproducing and publishing the code without following the license is a copyright violation. Instruction a computer programme to do the same thing should still be a copyright violation. Since this is the internet, I need to state the obvious: trivial code (defined on a case-by-case basis) is not copyrighted.
Books: the author expects you to reproduce their work, as long as it’s not another book. You wouldn’t get many sales if you sued someone for using the things you taught them.
The internet: actually the same as open source code. StackOverflow enforces a specific licensing clause to allow people to use code you provide. People who write posts that say “use this code to fix this problem,” are implicitly providing a license, too.
I don’t think using an LLM to generate code should remove its copyright but it also doesn’t remove the responsibility to not violate copyright.
What is transformative enough to be legal is fuzzy. At the end of the day, it only matters how the courts rule. I do know that the amount of actual mechanical change has less bearing on determining if the work was transformative. There are cases where you can copy 90% of a code base verbatim legally, and there are cases where no code is copied verbatim and it is a copyright violation. What you are describing most likely would not fly in court.
Humans will often fail to take responsibility for derivative works though when they don't know what work the output is derivative of, and how strongly.
And that's assuming they want to make clear what they are building on.
The stakes in the output go both ways: wanting to profit from IP and risking liability because of violating IP licenses.
What you're saying is already the status quo, but reality is that LLMs make it so easy to "paraphrase" code or simply .. ideas, too... so that the whole concept of IP is eroding. Because the sheer scale makes it almost impossible to distinguish specific things that cross the line from "learning from" to "copying IP".
I'm not really taking a specific stance here but acting as if it would be a solved problem seems hard to believe to me, especially since it is not a new discussion.
I thinl you could argue that the solution is that copyright or IP concepts are not working anymore, something many people already concluded before the LLM and AI explosion of the 2020s.
But that's not the state of the current law, right?
In addition, your claim that the user of an AI system is the only person who has any "stakes" also seems questionable to me, because after all, we are talking about output that the AI user already paid an AI company for to generate it.
> By this logic, shouldn't any code using an open-source framework or language also be open source? Even before AI, devs learned from Stack Overflow and plenty of open source code.
> Devs never just wrote code, they took ownership of the outcome. Business owners put their money on the line for product decisions.
Does the same thing apply to Mickey Mouse? Would you say it's fine to use Disney characters in my own creations, as long as I take ownership of the outcome and put my money on the line for product decisions?
I am not saying that taking responsibility gives you the license to violate copyrights.
My point is that devs have always learned from open-source code and the internet, then used that knowledge to write their own proprietary software. Why should it be any different when AI learns from the same sources and produces new code?
I think we need to go one level up in the law, and recognize that existing models of copyright and ownership simply don’t apply here, and come up with something new.
There is an undeniable core mathematical principle that should be recognized in law somehow: the emergent capabilities of LLMs are not possible without, effectively, the entire recorded intellectual output of human civilization since the dawn of recorded history as training data.
Trying to determine who “owns” that data, or each individual piece of it, leads us to insane distortions like LLM labs scanning and destroying books. Nobody, including the labs, thinks this is the best solution, but they’re doing it to comply with today’s interpretation of the law.
The reality is that if we are considering the entire corpus of training data, we all own it. We’re not really talking about who owns one book. We’re talking about who can be said to own all the knowledge in all the books. Who can be said to own the existence of books, or the conditions of civilization that cause us to value and preserve books in general.
The preservation and archiving of knowledge that enabled the emergence of LLMs is the product of a system of social relations that belong to the entire civilization. It is a commons. And when we talk about the future impact of these systems, we’re generally talking about taking the value of that commons, and concentrating the value of it in the hands of about a dozen people.
To me, there is no other way to think about a situation like that, where a tiny handful of people realize all the surplus value latent in the entire intellectual inheritance of human civilization, as anything other than theft.
One-off settlements with individual rights holders don't fix that. Neither does scrambling the output of LLMs so that they don’t reproduce their training data verbatim. I don’t think making LLM output non-copyrightable, or all LLM output open source, fixes it either.
I think we need to establish an entirely new framework for thinking about this stuff, and then go forward from there.
> The preservation and archiving of knowledge that enabled the emergence of LLMs is the product of a system of social relations that belong to the entire civilization. It is a commons.
This is the right framing. LLMs are a marvel built on the commons. Unfortunately, these wondrous information synthesis engines are currently captured and controlled by a tiny insane elite.
They deserve to be rewarded for their hard work, the risks they’ve taken, and the capital that was invested. However, the output should belong to civilisation. What should be open source, is the models themselves.
The problem with this is the assumption that models used are not a factor of the code already written by the user and ingested as context to determine the future state of the code according to the specifications.
All purely AI-generated code should be Open Source.
I've got plenty of technical designs that should not be Open Source and follow distinct design decisions AI can work with, not necessary know how to deal without context. If AI is additive it is just adding on the existing design not changing it.
Also I would dislike AI rot following into the codebase of Open Source projects making them unmaintainable.
End of the day question is if AI is driving you or are you driving the AI?
Doe v. GitHub already decided (I'm simplifying, read the actual thing yourself) that LLM output is not automatically open source - or specifically it doesn't infringe on the open source training material and doesn't inherit the oss copyright notice.
Thaler v. Perlmutter (again simplified) showed 100% AI output can't be given any copyright.
The directions the current cases are going are very much opposed to that idea and I wouldn't expect things to change in the opposite direction.
That is not the Thaler v. Perlmutter conclusion. That case was about whether the machine itself could be made an author under copyright law. That is very different than whether AI output can be protected by copyright law. I am not aware of any cases that test that question yet.
They will just claim the code is trade secret and use Defend Trade Secrets Act (DTSA).
And, the DTSA has only two reqs:
derive independent economic value from not being generally known or readily ascertainable by others, and the "owner" takes reasonable measures to maintain their secrecy (e.g., coke secret sauce).
since trade secret law does not prevent independent development or lawful reverse engineering, there are still opportunities. also, if a someone independently devs the same functionality, trade secret generally does not give them original dev ownership of that functionality. so, win.
Artcraft will be a good test case if Adobe goes after Brandon Thomas.
Much to be developed.
(and, this is why we keep extensive commit and development records outside of our genAI - ie provenance tracking)
I doubt that would hold the test of time, and any ruling that way will be overturned. The Berne Convention leaves what constitutes "authorship" as an exercise to the signatories, but the US probably has a lower standard.
US: "a modicum of creativity"
EU and UK: "the author's own intellectual creation"
If a court ruled in favor of AI authorship, that can't reconcile with the existing standard or with international treaties, and will likely not stand.
People who upload or download code from a US service also have to follow the law in their own countries.
Github also has to follow the law in any country where it allows usage of its services. Not all countries can enforce that but anywhere it operates or has enough paying customers can
Edit: open source software itself has only been lightly tested in court. But that isn't even my point here.
The US Copyright Office has assigned copyright to works that leveraged AI. The USCO stated,
> “[we] decided to register a copyright claim in [the] work because we find that it contains a sufficient amount of human original authorship in the selection, arrangement, and coordination of the AI-generated material that may be regarded as copyrightable.”
Copyright office activity is orthogonal to litigation and the court system.
If a work receives a copyright, the courts will treat it differently if it becomes the subject of litigation.
Even open source license claims have not been fully litigated. We don't have to wait on the court to make certain assumptions.
You can't apply naïve transitive reasoning to law. Law is not code. Your assertions are meaningless and unsupported noise until they're actually tried in a court.
a statement *about* the legal system can be true without being litigated *in* the legal system. Courts rely on precedent in addition to logical argument, so their rulings cannot be deduced from rational argument alone.
The US copyright office is not the body that owns interpretation of the law though. They don’t determine in court if an artifact can be protected by copyright law.
The USCO is the regulatory and administrative authority and will inform the decisions of judges. They provide expert legal opinion that carries substantial weight.
It would be a surprising result for the courts to overturn the USCO. The courts routinely side with the USCO.
What the USCO is saying is a temperature check on how this will play out.
fwiw, I suspect that a "low effort" AI-generated artifact will not be protected by copyright law. I suspect there will be a line somewhere, and it will be very unclear where it is. That said, I think it is important to be clear that this is uncharted territory, and while everything you said is true, the USCO does not establish precedent.
> It would be a surprising result
Yes, but also not the craziest thing the courts have ruled.
People show completely opposite reactions to AI generated creative contents vs technical materials such as source codes and instruction manuals.
If you think about the distinction between those, it is often so considered that both are covered by copyrights, despite technical definitions of copyrights never covering mere technical resources, such as some computer programming things and instruction documents.
Only contents of artistic values are supposed to be covered by copyright laws.
IMO, what AI revealed is that most copyrighted software and documents are in fact not covered by copyright laws as per the naive model of it among common folks as well as legal text, as opposed to what precedents say. Code is not copyrightable.
I don't necessarily support that stance from practical standpoint, and I'd rather like everything be licensed under AGPLv3++ than entire concept of code copyright fall apart overnight, but this feels like the writings on the wall to me.
Your first cited article doesn’t necessarily support the idea that agentic code generation is copyrightable.
“A Single Piece of American Cheese sits between those contrasting poles: the initial image was created entirely by the AI program based on a brief text prompt, then the user repeatedly selected elements of the image and used additional prompts to add/alter elements.”
Agentic development is steadily reducing the necessity for human involvement in the final output.
Every major corporation on earth is behaving as if it was. If/when it gets tested you can be sure that the pro-copyright side will be very, very well funded.
I think most expect the courts will land on training AI on copyrighted works is fine, as long as the output of the LLMs is transformative enough to not violate the original copyright. Exactly the same as if a human learned on copyrighted material and outputted their own artifacts.
This is very much still working it's way through the courts (in the US at least).
The first major precedent was in 2023 with Zarya of the Dawn comic book, where the US Copyright Office rules the midjourney-generated images were not protected, but the human-written text was.
I really have to wonder what context this person is operating in. The vast majority of software written in the world is proprietary, written for a company to aid some business process, and ridiculously specific to its use case, and freely intermixed with proprietary business information.
“Open sourcing” is not just a legal state, it’s an affirmative act that takes work to prepare the codebase for public consumption. Saying that all the weird business code in the world should be involuntarily publicized is an incoherent concept. What precisely would OP expect to happen in a world that adhered to their opinion? Because to be clear, 90%+ of all code being written today is LLM-generated, so OP suggests that companies should no longer have any privacy wrt their internal software, all LLM code just gets yeeted into the public. How could that possibly work?
Given how easy it is to write software now, this perspective seems to overindex on a past state of the world that no longer applies.
> Given how easy it is to write software now, this perspective seems to overindex on a past state of the world that no longer applies.
This opinion by OP also assumes that code itself has some form of value. With the ways LLM work now code by itself is nigh-on worthless and as the models are becoming more capable, the value of code keeps going down.
It's important to remember that the purpose of the license is to allow the copyright owner to exercise their rights in a manner they deem fit. Even if they choose to give the code away, that choice is captured in the form of a specifically worded license (MIT, BSD, etc.).
This means that licenses exist because code is copyrightable in the first place, with the license governing the use of said copyright. If AI code is not copyrightable, then no license can govern it.
> If AI code is not copyrightable, then no license can govern it.
This is the crux of the issue. A license controls usage of copyrighted intellectual property. Until the underlying copyright/ip issues are resolved, it seems a moot point to discuss licensing.
That said, if AI code is _not_ copyrightable, does it automatically go into the public domain? Or is there a third category?
> That said, if AI code is _not_ copyrightable, does it automatically go into the public domain? Or is there a third category?
It means that AI-generated code, including proprietary code, is not covered by the existing license mechanisms, including EULAs. Companies will of course handwave over this, but it would not surprise me if this is challenged in court at some point.
Maybe the Do What The F*ck You Want To Public License (WTFPL). Its a super-permissive open-source license that acts almost like a public domain dedication.
The distinction is between creation and distribution.
Even if you accept that AI code isn't protected by copyright, code isn't public domain unless it's released. The legal issue would likely revolve around what 'released' means.
Code that runs on private servers isn't technically released. Code in web pages technically is.
But ultimately legal decisions decide to a large extent on who has the most money.
The frontier labs may eventually try to argue that they own the IP of all code, content, and inventions generated using their technology. That argument may or may not stand, and it's more likely to be over patent rights than plain copyright.
But there's a fair chance they'll try it once AI is unavoidably embedded in everyone's workflow.
Copyright applies (or doesn't) from the moment of creation, as per international treaty in all Berne Convention signatory countries. Distribution is irrelevant. Otherwise the law wouldn't serve the original intent of preventing authors from being ripped off by publishers.
"Public domain" is a synonym for "copyright does not apply or no longer applies." There's nothing special about it beyond that.
Code without a copyright can generally be distributed and put in the pubic domain, and I wouldn't bet that the few existing court decisions won't be reversed.
But, the license reflects the rights that a copyright owner granted you; I don't see why laundering copyrighted material with training a model should remove the limitations on the material's usage that the author established.
So, while it makes sense to not be able to attach whatever copyright you want to LLM output, if it's the result of training on copyrighted code, the extent to which you're allowed to use it should be the intersection of the rights granted to you by that code's copyright owners.
It should thus carry a license that reflects the intersection of those rights.
What makes it much worse is that the model trainers don't tell you what specific sources determined the output, so you should probably just assume that it's the result of its whole training, and the only rights you have are those granted you by all the authors of the training material; and, this in all likelihood means no rights, for most LLM models in use now.
> It's important to remember that the purpose of the license is to allow the copyright owner to exercise their rights in a manner they deem fit.
Just as it's important to remember that the purpose of regulations like copyright and patents are to promote the progress of science and the useful arts.
We're not locked in to a suicide pact with what the Founding Fathers decided back in the late 1700s. Congress can decide that all LLM-generated code is automatically covered by -say- the GPLv3, even if no one holds the copyright to it.
If the absence of a particular person to hold the copyright twists your knickers, you could go a bit further and -say- decide that the people of the United States collectively hold the copyright to all LLM-generated code that's generated by US people or entities, and that relicensing of a particular piece of LLM-generated code must be agreed to by every US citizen that was alive when that code was created.
> We're not locked in to a suicide pact with what the Founding Fathers decided back in the late 1700s. Congress can decide that all LLM-generated code is automatically covered by -say- the GPLv3, even if no one holds the copyright to it.
I have no issue with changing the law. My comment is about the article and the author's misapplication of the law.
I think it would be the part about how courts have a fairly strong status quo bias and aren't going to remake the entire economy overnight with completely unenforceable provisions.
What are they going to do confiscate the world's gamer GPUs that can run this stuff all locally?
> you could go a bit further and -say- decide that the people of the United States collectively hold the copyright
Was going to comment something similar; we do have concepts of "dominium eminens", and "escheated property", and "public domain", and many other similar ideas. If the people (through their representatives) decide on a certain property regime, it absolutely can be implemented even if it's different from the status quo.
Maybe, but I don't think there's any legal president for that, is there? Much of existing law is based on centuries of private property rights. I don't see any government undoing that... or at least, not quickly.
> Maybe, but I don't think there's any legal president for that, is there?
We're not locked in to a suicide pact with what the Founding Fathers decided back in the late 1700s. Congress can decide that all LLM-generated code is automatically covered by -say- the GPLv3, even if no one holds the copyright to it.
Article I, Section 8 of the Constitution and its various subsections pretty clearly gives Congress this power.
The IP laws are a) not really modelled after material property rights; b) are quite much more recent than "centuries"; c) there is legal precedent for that, see e.g. the airplane-related patents being pooled in 1917 by the demand from the US government.
Nobody is going to interpret the laws that way. That's completely stupid.
Everyone is using AI generated code in all sorts of ways, and the idea that the courts are just going to say "nuh-uh. All existing use of AI code is illegal" is completely ridiculous.
Courts don't upend entire economies like that. This would be the remaking of the entire software industry overnight and it would be completely unenforceable.
I'm certainly not going to listen to any such ruling and will continue making code locally. As will basically every other software engineer in the industry.
> Nobody is going to interpret the laws that way. That's completely stupid.
You're absolutely right! And a little more than two hours before you posted this comment, I posted this [0] one! Make sure to read the parent comment for the context... it's pretty important.
Gotcha, not really related to my point that the courts aren't going to completely remake the entire software industry over night and neither is congress.
That's absurd. It is a fantasy. Neither lawmakers or anyone else is going to remake the software economy.
Your point was entirely off topic, and the retcon of your intention is weaksauce.
> It is a fantasy.
Reread the title of TFA.
> Neither lawmakers or anyone else is going to remake the software economy.
LLM-generated code is -when measured in regulatory timescales- brand new and is -according to boosters- "remak[ing] the software economy". The time to make new law and regulation to make it prosocial, rather than profoundly antisocial is _now_.
Kind of a silly position. I mean, if it's you're code verbatim, then yeah, it's a violation. Shouldn't be allowed. But like... it saw open-source code and therefore all output must be open source? I don't know about that.
Here's where I can get on board. I think people should be putting AI to work to improve everyone's lives for free. Your version of "should" is like "we should make this a law." My version of "should" is more like a moral recommendation -- we should be using as much of our subscriptions as possible to benefit everyone instead of enriching a handful of people. I view it as a natural extension of the spirit of free software to begin with. Software should be free (as in freedom), and we finally have enough development power to make real headway on that.
I have some projects I've been working on over a year. Sitting down working making design decisions. Research. Using ai doesn't mean I did no work. It doesn't matter if the ai was trained. I still did work.
I don't understand this view point. Using ai and making high quality stuff is not pushing a button. Its hard. It takes real effort and diligence. And you have to pay actual money out of pocket to deliver it.
What if I am creating something that does not exists yet using AI? So i did the research, layout the architecture, decide on the implementation details, write the documentation and the rules AI has to use to create the software, because I am only one person and I can't physically write all that code on my own on a timely basis (it would take me years to write all that code, test it and debug it until i am satisfied)? I am creating something new, not copying anything, so where I stand? Can i publish my code, say with MIT licensing, and retain the copyright?
> I have some projects I've been working on over a year. Sitting down working making design decisions. Research. Using ai doesn't mean I did no work. It doesn't matter if the ai was trained. I still did work.
> I don't understand this view point. Using ai and making high quality stuff is not pushing a button. Its hard. It takes real effort and diligence. And you have to pay actual money out of pocket to deliver it.
You seem to believe that because you put months of work into this, you should have rights over the result, right?
But once you release, anyone can spend a few dozens of dollars and replicate it. Do they get the same rights over the clone that you got over your product, even though you potentially spent months coming up with it and they cloned it in a day?
There's really only two answers to this - either you agree that they should get the same rights over the clone that you got over your product, or you don't agree that they should have the same rights over their clone that you have over your product.
In either case, you are not better off:
1. If they get the same rights over the clone that you get on your product, you can't really stop people cloning your stuff, adding a feature or two, removing another feature or two, and then release something better than you for 1/10th of the cost of production.
2. If they don't get the same rights as you did because they used your product as training material for a clone, then wtf do you get any rights at all, after using millions of other people's products as training material?
I don't see any scenario where there is an outcome beneficial to you. Do you?
> This is a false dichotomy. In this example the person you’re replying to isn’t wholesale copying someone’s work.
So? I said "clone", but clarified that features may be added to the clone to make a better product.
Taking his product $FOO, adding it to my training set/context/RAG/LORA and/or tool call to explore his product in detail before I tell my LLM to produce $FOO+1 is exactly the same thing that he did.
3. Others are free to use the produced material as educational/training material if they can gain access to the code to then use it for their inspiration. But they are not allowed to outright duplicate it.
This seems the obvious answer? No? The exact clone at 1/10 the cost is just really unlikely imo, especially if it's closed source. And with open source this was already a problem anyway, you could always fork, but hey, turns out most people just stick to the original anyway
> No? The exact clone at 1/10 the cost is just really unlikely imo, especially if it's closed source.
The availability of the source is irrelevant; I posed this question with the understanding that I could simply add your product, whether closed or not, to my LLM, and get it to produce a better product without all that pesky effort and token usage you used.
Release your product. See how quickly it gets "cloned". i.e. copied and reproduced without anyone needing the source code.
All that effort of yours... just to result in someone else producing the same product in a different language, but with all your product decisions already baked in for nowhere near what it cost you.
> And with open source this was already a problem anyway, you could always fork
Sure, and people famously did so, but they could not clone original closed-source products without a lot of effort, often taking months to years. Now, the result of you spending years on perfecting a product means that a clone can arise in a matter of days.
It also takes effort to copy something, right? Say I made a copy of a book by hand, does my effort make it any less of a copy?
Say I am stealing stuff that’s really heavy, can I demand I earned it because it was a lot of effort?
I am not trying to be facetious. I also question property rights because someone overpowered someone else to make that property theirs at some point in time. Whose effort is earned, and whose is robbery?
As far as I can tell we picked an arbitrary time after which all forced acquisition is considered robbery.
That being said, open source is for everyone to benefit. Violating an open source license is sort of a crime against everyone, not just the creator.
Absolutely. There is so much variability in the code that can be generated for a particular purpose. I do hundreds of iterations before my LLM-generated code reaches the desired shape. The built harness, the written AGENTS.md and the iterations affects what is written greatly.
And at least of the level of feeling the answer is that it is my code.
It's unfortunate that you had to pay actual money and effort, but there was also actual money and effort involved in the projects on which the LLM output is based.
When a carpenter crafts a chair, the tree first had to make effort to generate the wood.
Because chairs are in effect an open source design, does that mean carpenters shouldn't get paid?
Its nonsensical.
I think that people who believe these arguments have bought into ai hype too much and genuinely are fooled into thinking pressing a button to spit out boiler plate code somehow equates to a genuine saleable product. It doesn't and it won't. And that stuff shouldn't be open source because it's a waste of everyone's time to consume.
The difference between the input to the carpenter and the output of the carpenter is significantly greater than the input from you to the LLM, and the output of LLM. Majority of the alterations to the wood by the carpenter produced the chair. The words you typed into an LLM are less than 0.000000001% of the total inputs that it took to create its output (because training data counts, and it’s a lot bigger than anything you may have typed)
And however hard a carpenter might work on a chair, if he copies a protected design he can't sell the chair.
However much work you might add on top of other people's copyrighted code, you don't get to sell it and use it however you want if you don't have the permission of the original authors.
> However much work you might add on top of other people's copyrighted code, you don't get to sell it and use it however you want if you don't have the permission of the original authors.
It gets even more hilarious when you realise that the GP (tim) does not consider that, if you make the output of AI that was trained on copyrighted material legal to re-copyright, then it applies to him as well - all that hard work he did can be used chucked into an LLM either as context, or via LoRA, or even via simple explorative usage via tool calls, and then regenerated from that LLM, taking away whatever copyright the product had by generating a competing clone.
Just because AI was involved doesn’t mean significant human thought and effort didn’t go into the production of those artifacts.
My agentic software development loop requires heavy involvement of real-world feedback. The ratio is huge. 20 mins of agentic software coding against hours or days of testing, trying it out, real-world contact, thinking about what feature is next or what tweak would make it more usable, smooth, intuitive. That feedback is baked into the product. Forcing it to be open source steals all that time and effort for me, and takes from
me the decision as to how and whether to open-source it. So you mean the code that I’ve agentically written to help manage my private financial affairs (on local models, for privacy) or my kids’ complicated school schedules must now be open sourced?
Furthermore, copyright protects only the expression, not the idea - reimplementing the same idea with "different words" has always been permitted under copyright law (patents may protect the idea, though).
Of course, the fact that it would lack copyright protection it doesn't implythat you have to do something with the code for management of your kid's schedules, just that if someone else somehow legally gets hold of it, the law won't prevent them from making and distributing copies or derived work.
> Forcing it to be open source steals all that time and effort for me
You are the one "stealing" the time and effort of thousands of people who have written free and open source software. If the ratio really is so huge, then just cut out the small part that does all of the stealing.
> So you mean the code that I’ve agentically written to help manage my private financial affairs (on local models, for privacy) or my kids’ complicated school schedules must now be open sourced?
Open source licensing is only relevant when distributing the software, if you just use it privately it has no effect. You are not forced to make it public.
> My agentic software development loop requires heavy involvement of real-world feedback. The ratio is huge. 20 mins of agentic software coding against hours or days of testing, trying it out, real-world contact, thinking about what feature is next or what tweak would make it more usable, smooth, intuitive. That feedback is baked into the product. Forcing it to be open source steals all that time and effort for me, and takes from me the decision as to how and whether to open-source it.
It doesn't need to be open-sourced: if it's legal to take an LLM that was trained on every other product in existence and use that LLM to produce clones of all those products, then it's legal to do the same to your product to.
So, go ahead and release. There'll be an open-source clone in no time if the product is any good at all.
In fact, if there are no clones, take that as direct feedback that you wasted your money on tokens - your product is so bad no one even wants it for the few dollars it will take to clone it.
I don’t know about one-shot-prompt toy apps, but when it comes to building real stuff with AI, it still takes significant effort from devs to get the results we need. There’s a difference, right? “Give me the best result” vs “Give me the result I want”. Most serious projects, at least from what I know, run on the “Give me the result I want” principle, and that’s where the dev effort goes. I know the future is heading toward “Give me the best result with no mistakes”, but that changes how much effort it takes, not who owns the output. Even if the code is generated by AI, humans own the result, because they worked with the machine to get the result they wanted. Same was true before AI: humans worked with the machine to get the result they wanted. It’s just way faster now. Using a machine to build something has never meant you don’t own what you built.
The other take is AI code generation machines are directed by human inputs and will never start without that human input, so therefore it was created by a human.
Much like how a photographer 'just' clicks a button on a machine and through a very complicated process it creates media, so does the human who makes media via an AI machine.
I think because of how corporate power works and how much money is at stake this will end up as the end result. Laws are made up by humans and a majority vote in a parliament can set this to be so, previous rulings be damned. And most treaties never really ever think about how media was made because this was never an question until recently. Copyright are not constitutional rights in most counties.
Actually: All AI-generated _Output_ should be Open Source (not just code)
It's the (open) knowledge of humanity, and the thin layer above is only a minor fraction (0.1 if at all) of the 'product', which should not be allowed to be privatized by anyone - let alone destroying originals (like books) to own 'it' !
This article is mostly ragebait, and it's claim is hollow when looked thru the lens of computing history.
Let us skip the whole Bell Labs C/C++/unix/sh/ksh -> BSD csh/vi/bsd/tcp/ip, and other proprietary unix variants/utils and focus instead on today's landscape.
There are two axes we can partition the whole ecosystem. Free/OpenBSD, Clang/LLVM, musl, busybox, X11/Xlib, sqlite, postgresql, nginx, Python, Go, Rust, many embedded rtos, vendor sdk, utilities etc.. and then there is Linux/GCC/glibc, etc. Along the licensing spectrum there is GPL and then the rest of the permissive licenses MIT/BSD/Apache etc where source code publishing is not a requirement.
These permissive license offerings are more prevalent compared to the GPL offerings and as such one can gain all the knowledge needed to build a fully functional system. LLM could memorize, or copy the pattern, so even though it is not a clean room, courts seem to be ok as long as things are not spit out verbatim.
Add to this, the fact the there are tons of research articles, user manuals, RFCs, so one can learn and understand how to build the needed reusable component if small and give their own prompt (and bugs) and and let LLM do as user described.
In the US, purely AI-generated code isn’t copyrightable, but it also doesn’t mean you have to publish it. Also, putting an open-source license on code nobody holds a copyright to has no legal effect. So, ethically or otherwise, there is no strong and convincing argument that open sourcing is the right thing to do.
What 'creators' could do is to compile a list of all the frequently used permissive licenses that are applicable to their system and just make it part of their software license attribution.
Whilst the first training sets would have used Stack overflow and open source projects.
I have my suspicions that the AI companies have been using reverse engineering to gain more training data.
Ethics don't really concern these companies so...
When you have essentially infinite compute and are constantly looking for more training data. To me the next logical step is to start reversing compiled programs.
It could also explain why these AI models are so good at it too...
> AI models were trained on a massive corpus of Open Source
using open source shouldn't require the final product to be open source. that's pretty much most of proprietary software. most of it is built using open source code but the final product is due to a combination of thoughtful product and software knowledge and must be private since it's really the value you produce.
> purely AI-generated code is not written by a human, meaning it should not be copyrightable.
you can say AI generated code is based on human code, and that human code is open source. depending on the license (likely MIT) its technically legal to choose the generated code to be private.
> To make things fair across all jurisdictions, and to honor the open ecosystem that made AI possible, developers and companies should explicitly release AI-generated code under Open Source licenses or dedicate it to the public domain using the Creative Commons Zero (CC0) license.
i somewhat agree here. AI generated code should include the original license of its authors unless it's purely AI generated using original variable names.
i previously released AI generated products/software as open source but come to think of it, in the world of abundant software, i believe there's more reason to actually privatize your code vs putting it out in the public. especially for paid software products.
Not the AI's source code output should be open source but the prompt should be. The prompt is the real source and can be used in the future with newer LLM models.
> purely AI-generated code is not written by a human, meaning it should not be copyrightable
There's a lot of room in between code that is closed-source and code that is "copyrightable".
Should code crafted via cut-and-paste from StackOverflow be open source? How about newsgroups before that? What about apps that use "no code" tooling? Is there a threshold for the use of external libraries and packages that cross the threshold for "that code was simply scraped from the internet and used without permission or proper attribution"? How about code I wrote while using auto-complete a lot?
I appreciate the author's sentiment and recognize that LLMs have aggregated the work of many, many people. I'm guessing their work wasn't wholly original either, like so much of the code I wrote before the age of LLMs. Whether those people whose work has been aggregated to train LLMs own that work - those ideas, those thoughts, those strategies - is exceedingly complex and (I speculate) largely won't hold up to real scrutiny.
I mostly agree and have pretty much every one of my projects open source. One in particular, though, is not. And that is because the design of it is heavily domain specific and is built on loads and loads of experience. Not sure how to reconcile that...
There's a difference between something being released as open source and something that never gets released at all. Open source or not only becomes relevant when making the binary available to other people.
If you type in something like: the history of code and copying previous code versus learning from it and applying that to creative intent behind AI usage for code creation
--- you will find that historically that has been a progression from day one. The effort to get good code out of an AI and test/validate it does not minimize the creative intent of the user. Now intent is the operative work. Some intent is bad, some is good. Determining intent is hard for humans as it is for AI. I don't think there will be good answers except for individuals. The courts may rule one way or another. Public opinion in different groups will sway one way or the other. Saying anything is this or that is a slippery slope. The whole premise of this small article seems flawed as a concept and just ignorant of the history. Much like suing someone for a musical phrase of a few notes. There are the same chords/melodies in countless songs and it is not done maliciously. There are only 12 notes, so you figure it out. I think everyone is going crazy about AI on both sides. There are a large group in the middle that are learning how to use it and what it is and is not good for.
I’m confused. Is this article meant to be a scholarly legal argument (albeit, by a novice)? Or a normative statement about what the author would prefer?
It certainly doesn’t seem to be about practical solutions. It doesn’t even touch on the ramifications of its suggestions. I’m pretty sure that it would be a massive boon to data center operators and cause most of the readers of hacker news to lose our livelihoods.
Personally I am less concerned with all AI generated code being open source. We will very quickly be drowning in a sea of code, and who even wants to see it if it's easily re-created by AI? Eventually everyone will be using AI and I don't think it's feasible or makes sense for all code to be open source. Personally I _would_ like to see all models trained on public data become open-weights. To me it seems unfair to let a few companies lock up all the value of the collective works of civilization without providing something back for free.
Two points I think making AI-generated code opensource
Point 1: AI can (or will be able to) reverse engineer all code from how the program works. Thus, there seems to be little point in opensourcing AI generated code because it can just be reversed engineered
Point 2: Correct me if wrong but can AI generated code actually be copyrighted? If someone just keeps prompting but not touching the code would this still be considered pure AI generated work? If this is the case then point 1 and point 2 basically means AI generated code is already open source in effect if not in name.
A lot of y'all here are missing something important: open source code is an application of copyright law, and it has already become clear in some countries that the output of an LLM is not coverable under copyright (i.e. it is public domain).
In the UK, the output of an LLM can come under copyright and therefore the author can license it as an open source product, but in the US it can't be done.
Since last year everything I was generating with AI I put it opensource on github! Some of these tools I use on daily basis others were experiments. It felt natural to do so, I never for a second felt or thought I owned the code output of LLMs even if I am the one who prompted it
> As soon as I start refining it, and commanding the AI to fix the mistakes, this argument starts to lose ground fast.
Definitely. And if the result isn’t essentially a clone of some existing copyrighted work, there really is no viable argument under current copyright law.
At this point, since LLMs can reverse executables anyway, making something Open Source is hardly any leverage. Instead we could say that any products generated by AI face a tax.
Actually, why not go straight to the source and tax the AI companies for the public benefit. That is the logical reduction of the argument.
He’s not wrong, but I would argue a better model is the Derivative Work. This already exists in copyright law.
If I write a space opera that is a LOT like Star Wars there are major legal implications to this. If I write an original work of science fiction involving a weapon that destroys planets as an allegory to the history of the atomic bomb, I’m probably in the clear even if one of the characters is a furry alien with a laser sword.
The challenge now is how to apply that framework when nobody can see the original or resulting code.
Derivative works still have their own copyrights, though. For instance, Kouta Hirano might own Hellsing, but that doesn't automatically mean that he owns enthusiastic walks through the woods, "bitch I eat people," or High School of the Dead.
By that logic, LLMs that were trained purely on proprietary code should be allowed to generate proprietary code (Microsoft, Oracle, Apple, etc. should have a quite large codebase when combining all their products).
And then again, I wonder which license should be used. Is this a case of copyleft, and therefore everything generated should be something like GPL, or is it more like public domain because no copyright can be exercised?
Alternative view on the problem. if code is really a solved problem , open source becomes a security problem? with all the attacks happening mostly on open source projects, it is not farfetched that a secure bank system will be ai generated with core libraries implementing everything from scratch, including runtime. Big business, big problems.
This idea of no libraries/external dependencies sounds extreme, until you got hacked by a tiny library for http calls and that cost you billions of losses.
What's wrong with this website? When I open it in Safari on macOS, it consumes all my RAM. 20GB to start. I got a system warning to force quit some apps to maintain stability (which was a first in 5 years). When I closed other non-Safari apps, Safari usage then went up to 30GB.
Quitting and relaunching Safari, then returning to the site, produces the same results. The process for this website is what is using the RAM, it persists after closing the tab, and needs to be force quit.
I disagree, however companies who decide to work on proprietary code need to be doing comprehensive license similarity scans on their codebase or risk licensing issues
Definitely agree with that in concept, but there really a fact that nobody realized until recently after all the game decompilation/mix-ups: LLMs can read assembly fairly well and traditional obfuscation techniques doesn't really work on them, so if you have access to the binary and a disassembler, an LLM agent will be able to reverse engineer and reconstruct the likely high level source code of most software given enough time anyways.
In other words, almost nothing is closed source to LLMs if they have access to the binary.
The article is right. If no human effort is involved then the code should be open source. The article also implies a very detailed line too. It doesn’t say that the human effort part can’t be copyrighted or intellectual property.
As an example, I have developed a set of grounding guidance for my AI models at my work. The specific grammar and vocabulary belong to my employer. But I also have a clean-room version of it that completely strips out that proprietary infringing information while still retaining a similar semantic structure. That will be posted next month on my GitHub under a GPL-3 license like I do with my other non-infringing AI-assisted work.
In a way all LLM code is already open source. We all have open access to the system which generates the code. All you need is the prompt, so that is the non open element.
> developers more and more rely on AI without even reviewing the generated output, let alone modifying it to fit their specific needs. Code like this, where humans put little effort, except writing prompts, should definitely be Open Source.
A person's contribution here is difficult to measure. This statement reminds me of one of my older bosses who priced software per lines written.
An AI-generated software's quality reflects the author's vision, taste, and expertise (if any). These are there regardless of the number of lines reviewed.
I would've supported open source for completely AI-generated apps including the conception of the idea.
I'm all for the concept of all code being open source... but I would then need a way to protect my final product from just being cloned or ripped off by a competitor.
With code itself being so cheap these days, it's the final product that has value. The requirements, architecture, usage data, the hundreds of meetings to figure out what customers want/need, the marketing, etc. A lot of the learnings and most of my business are expressed in the code though, so making it open is a massive risk. The code on it's own I don't even think is valuable. The only ones who'd benefit are competitors and upstarts targeting my customers.
I agree, but also, any code that a company can generate and use, an open source solution can generate and release, effectively eating the corporation’s lunch in many cases.
> The second reason is legal: purely AI-generated code is not written by a human, meaning it should not be copyrightable
I think that's the actual point here. AI-generated code shouldn't necessarily be open source, but it is not (or should not be) copyrighted. You can still keep it as a trade secret, but you have no legal protections that prevent others from reusing it once they have access to it.
PLEASE stop right there. Nations are sovereign, they decide for themselves if/how far copyrights are enforced. Let's keep it that way. There's enough [800lb gorilla meddling in other's affairs] as it is.
I could not disagree more with this view.
Some of the works that reasoning models were trained on were already in the public domain.
OP's statement is an attempt to get hold of code that they never build themselves.
It takes human effort and human input - even if all it only determines is merely an initial random seed value - to generate the code in question.
There is no fundamental baseline and there cannot be such a baseline that would provide for such a strict licensing of AI-generated code.
It seems pretty clear-cut to me. AI companies plundered the commons, but the commons have licenses that come with them that clearly state that their use and derivatives thereof must also include the license and be open source.
This is a chance for great good for humanity and for the AI companies to do the right thing for themselves and for all of us.
Not releasing the code as open source would be just... well... evil.
And a further question arises for me: If we don't start sharing the fruits of the machines now, then when?
Valid point but this will happen due to market forces rather than moral reasons. Anyone can write code now using AI, so charging for the code itself is getting harder every day, and only the trustworthy people will get the distribution.
The rest of the code will be open source, see how many open Wispr flow alternatives are there already, everyone has same features.
I agree with the spirit of this. I'm a longtime open source developer and generally prefer GPL for my manual code. For my AI-generated code I'm using a multiple choice license of Apache, BSD, GPL, MIT.
If you're curious, here's my newest open source which helps build Rust web services along with Axum: https://github.com/publicprojects-io/axa and you can see the LICENSE.md file.
Not if you see the AI-generated code as same as 90's java bytecode compiled by a tool (machine). In both cases, the input (AI prompts or 90's java code) is human-written. Just because a machine is involved in converting human input to some next low-level artifact, the low-level artifact does not become open source, when the human input is not.
I get the sentiment, but how would you prove a negative? You can get "caught" using AI, but not having been caught does not and can not prove you didn't. Given that 99.999% of projects now use AI to some extend, the logical default assumption is going to be you are probably not in the 0.001%
Open source is not always open source as we all know there are so many different flavours of it. How do you make that distinction when using AI to generate your code?
This is a debate that we need to have before it gets event more muddier.
Related question I just thought of reading this: if you are an employee at a company that you know 100% AI generates its code, and AI-generated code is not copyrightable... is that employee legally allowed to leave the company taking a full copy of the source code, and set it up as their own business?
I use AI to develop apps for manufacturing but I still have to think and point AI how things will work together. If I simply tell AI to develop something complex most of the time it will do a really bad job doing so. I do have a coding background but with AI I can focus on other things and use the time I’d spend coding to work on other problems.
I kind of agree, but like most software engineering takes I think this misses the point of software as a whole.
What the software _does_ is almost always more important than how it does it (save for some amazing new technique which is very rare).
There are only so many ways to write a binary search or an octree, but the overall idea behind, say, doom is unique.
Open sourcing big blocks of code generated using standard techniques isn’t that useful (since you can just generate the same ones yourself), but the concept behind the application is the important bit (which of course, is now also easy to just generate).
If anyone can just generate their own version, who cares if it’s open source?
Fully AI output from a single prompt is not copyrightable (in the US). Is fully AI output from a sequence of prompts not copyrightable? If it is, at what point does this change?
> Some technologies allow users to provide iterative “feedback” by providing additional prompts to the machine. For example, the user may instruct the AI to revise the generated text to mention a topic or emphasize a particular point. While such instructions may give a user greater influence over the output, the AI technology is what determines how to implement those additional instructions.
No offense to anybody but I do believe there is no Intellectual Property (IP) anymore in the age of AI if one wants to defend or attack it was breached.
All models should also be open source and socially owned. All of this output is the culmination of human knowledge and should be treated as such. I get this is about copyright but I don’t think people are thinking big enough with this stuff.
There is so little value to most code produced with models these days I just don’t see the point.
The real value ends up being found in software where there is a person behind it continuously refining it into something others will look at and say “Someone clearly paid attention to this.” Models can’t (yet) reproduce that because they can’t/don’t go interacting with users or asking bigger picture questions.
Whether someone is producing that software with a model or not doesn’t change much about the ownership picture to me: it’s the person that owns the project, and the LLM is a tool.
Once again, it all comes down to it being a tool and how it’s used, and there are no useful blanket statements that can be made about it (or how the tool’s output should be classified, legally or otherwise)
You can't? Do people really think software is just prompt and done? People seem to forget software isnt just writing code, its solving really problems, and architecting. Coding was a small part.
Looks like the author wants to abolish copyright in a very round-about way.
They are saying that AI-generated code should be released under CC0 due to complexities around the status of software copyright in various jurisdictions.
But if the concept of copyright was abolished as a whole, then licences such as GPL would also lose their power. Some people say that's a negative thing, but I do not necessarily think so, because it achieves the stated goal of the GPL, software freedom (i.e. you don't need to be granted a licence to copy and modify things)
No, the four freedoms rely on copyright. If there is no copyright, any company can take the open source code, modify it as they like, and never give back the modifications to anyone. It's the equivalent of an MIT license (technically, it's more like public domain but let's ignore this detail here). That's fine for many but far from the goals of the GPL.
No copyright doesn't mean you can't keep your source code a tightly kept business secret. You can even combine it with NDAs and contracts to re-institute something akin to copyright.
> First of all, AI models were trained on a massive corpus of Open Source code written since RMS announced the GNU Project and the Free Software Movement.
I too have trained on the massive corpus of open source code. Should everything I code also be open source? Or only if I leverage an artificial neural network as a prosthesis to enhance the capabilities of my natural neural network?
I'm claiming that my tools are extensions of me and parts of me. My freedom of expression should include the use of my arms, legs, printing press and language models.
Makes sense, many of the programming languages are open source, other novelty things were released as open source aswell, we as programmers/software engineers does a bunch of our solutions based on public code or under Creative Commons.
What I agree with the post is specific licenses require us some specific requirements like release the code if it's a derivation of prior work!
I'm tired of these child like responses from adult professionals in this industry. Do you argue just for the sake of arguing? No, your exposure to opensource code wasn't "massive" and you are clearly smart enough to know this but you had this make this comment as rage bait.
An entire generation of web developers learned by viewing the source of webpages they visited at a time when HTML and JavaScript were still nascent. I don't think setting the bar for "massive" is the way to win this argument. It's valid because the biggest difference between an LLM reading Stack Overflow and remembering content and me finding answers there and learning something is that the LLM can do it much, much faster.
The difference is, just as a child doesn't need to consume the entirety of a language before being able to communicate with said language, a human developer doesn't need to consume all of Stack Overflow, Open source projects etc. in order to start producing functional work.
I'm confused - I thought you said the response was child-like, not that a child can do the work. I'm not advocating that an LLM has child's level of experience, just that humans learns how to do things from looking at previously done work, so why is that not an allowed method for an artificial intelligence? How else is it supposed to learn?
This also ignores all of the great artists who learned by sitting at the feet of other great artists. Sometimes they extend the art of their teachers, sometimes they diverge.
Do you people understand that my comment was about the parent commenter exaggeration on how they were "trained" on a "massive (OSS) corpus"? I'm sorry I hurt your sacred cow by implying LLMs need stupid amounts of data.
So if future models are able to generalize better from much smaller amounts of data, you think that all the current concerns disappear? It doesn’t seem like a very good argument if it’s purely based on the amount of data needed to train a model.
What matters are things like where the data comes from, how similar the output is to the input, how close the result is to existing (copyrighted) work, and so on.
Simply using copyrighted work, even all copyrighted work, to train a model doesn’t, by itself, trigger any legal consequences. In that respect at least, the comment you called “child like” was correct.
My comment was simply about the parent comments exaggerated claim. I do believe it should have some consequences, what those should be I am not qualified to decide.
I have learnt to read by training on a lot of books, I should then open source all future things I write. I've learnt to walk by watching other people, I probably should publish videos of me whenever I walk.
What problem does this solve exactly? I think when most code becomes generated it’s all from the same hivemind of the human collective. I see arguments that code is not the hard part of development, so why are we treating it so sanctimoniously now?
I completely agree with the OP. What irks me the most is that Amodei has trained his model on someone's else IP and now he wants to resell this to people and has the nerve to complain about distillation. Distillation should be god given right. I guess that is why they took down Jensen.
There is no such thing as AI generated code, just as there is no such thing as keyboard generated code. The only difference is that your keyboard manufacturer never had the gall to sign git commits -qwerty.
The answer is obvious. If you want to protect your code, invent a new programming language that does weird things that make it difficult for an llm to learn.
Joking of course. Not sure copywriting code ever survives the AI era.
Essentially you're saying: AI companies are allowed to steal from all of us in order to make money, and in return they are allowed to read our private conversations and train future models in order to make even more money.
couldn't you always generate infinite code? just write more of it. but what open source lets you do is read the code for a particular piece of software, to see what it actually does. generating infinite code has nothing to do with that.
you could also do this before, but only with infinite time. open source value is shifting. open source is human-validated code. just OSSing generated code has no value.
If humans are directing AI to generate code, it cannot be open source unless the human decides to make it. It is their vision that AI acted upon. So, i dont see the logic in this post or thinking. this sounds like an another altruistic progressive california thing
Maybe by the same logic all code should be open source before AI if the author used stackoverflow, read a book, remembered their college education or read a blog?
Good idea. We just have to pay for the human-written part. Same with medical research and new medicines, new energy technologies, robotics, movies and music, and future AI. Eventually, as AI does more and more, everything will be free and open source.
This article makes no sense. None of the arguments are sound and conclusive. First argument: Non sequitur. Programmers were also trained by open source software and from stack overflow and stack exchange. That's not a sound reason for their programs to be open source. Second argument: AI-generated content is not copyrightable. If that's the case, then it cannot be open source since open source licenses rely on copyright for their enforcement. The author confuses public domain with open source. Public domain and open source are two completely different types of categories with different legal implications. Third argument: This contradicts the second argument. Now programmers using AI should publish their code under an open source license or under CC0. That's a demand. Not everybody will do that and no argument is presented why they should do so. Fourth point: Not all code is a hundred percent human written and a hundred percent AI written. The author concludes from this problem that "Code like this, where humans put little effort, except writing prompts, should definitely be Open Source." There is no argument here at all. It's another personal demand without justification.
At least, the author should try to back up their demands with further justifications and make them less contradictory.
You cannot enforce that it's available without copyright and a corresponding license. I thought the author meant the code produced by AI should also be available to others somehow but perhaps I've misunderstood.
WOKE, no other word fits. You should move to another planet.
Everything is stolen online, from marketing copy to books. And code is the one fight you want to have, the thing actually that is productive.
I’ve been thinking a lot lately about how LLMs have essentially made all code open source in a strange and very capitalist way: They’ve appropriated our human work—without regard for any licenses—by force and without any compensation to the authors, and are now selling the code back to anyone who’s willing to pay for it.
Following this logic, there should be no fees at all on coding models, since they scraped the web on this knowledge. Same applies for books, pictures, music btw. This debate has not even started yet.
I think copyright law has simply failed to keep up with the reality of technological progress.
I don’t really see a problem with LLMs slurping up and regurgitating the world’s knowledge, if it was all freely and equitably available, and more importantly had been beforehand. We’d all herald it as an incredible innovation. And in some ways it is even now, because I do have access to it in numerous forms.
The problem is that some author’s livelihood is adversely impacted by this, as copyright is (in a sense) what ensured their remuneration for the original work. So LLMs undermine the existing social and financial order in deleterious ways. It’s the same thing we’ve seen play out in mathematics, a bunch if tech bros show up with a powerful tool, use it in a brash manner that likely causes real harm, then worse, start pulling up the ladder when big media conglomerates complain.
I suspect we’ll soon be in a situation where we no longer have easy access to unfiltered LLM outputs. If your OSS project was in the training corpus it will still retrieve it. It will not, however, retrieve content from NYT or other protected sources. The end result is that small-scale, solo creators are damaged, and large-scale creators are still afforded copyright protection.
There is no way to enforce opennness. The LLM gives me output, I iterate on it by hand and with other language models, who does it belong to now? We are ushering in a post scarcity world for a select few.
No one goes to an LLM and says, 'Make code'. More often than not (and with exceptions) there's a very specific problem getting solved. The distinction between human 'effort' (whatever that means) and Ai effort (??) is more or less arbitrary... Infact, it's almost xenophobic and snooty. Pre-Ai humans had other humans build something like x but with y specific features. If it wasn't a problem then, it's not a problem now just because people don't have to directly deal with another human. In the same vein, I think people are forgetting that sometimes projects are birthed by slapping together code (or an instruction for code) from all across the Internet.
So really... What even is 'purely Ai generated' code? Nothing comes from nothingness and generally I'd prefer the purest expression of my ideas be protected regardless of who wrote it... Human4hire or Chat.
EDIT:
This article is unnecessarily upsetting. I don't even understand the problem when at the end of the day. A shit ton of the value accrues to all of us... humans! Not even the companies (atleast not so far). If Artificial Intelligence really is a utility, like electricity, then wtf is exactly the problem? I hope nobody really gives any real thoughts to policing Ai usage... It'd be as unreasonable as policing a calculator or pen and paper.
Code has lost almost all of it's value, who cares if it is open source. In a couple of years, you will say "make me a web browser" and 4 hours later performant, fully standards compliant web browser will be running.
The argument between open and closed source has little value anymore and will soon not even make sense to discuss. It will be like writing "2 + 2" and arguing over whether my "closed source" copy of "2 + 2" vs your "open source" copy of "2 + 2". When something is trivial to reproduce it becomes absurd to argue over who owns it.
And everything that can be handled as a common should.
That is everything bar one own mind and body, I guess, but your lineage might vary, and for sure some people out there would rather be deemed owner of everything under the sun, starting with human corps and souls.
Until and unless every creator and contributor of every piece of training data consumed by these models is fairly compensated for the use of their works, then absolutely everything created by these models should be 100% open source - including the models themselves.
It is disappointing to see many people who espoused that copyright is bullshit now saying copyright should make people do what they personally want. I suppose it was never about freedom for them, only getting what they want when they want it.
Whether programs written by LLMs should be protected by copyright law is also an entirely different question than should the authors be legally required to release the source code under a specific kind of license.
This article misunderstands copyright law -- citing an example that argues against it.
It cited Naruto v Slater, which unsuccessfully argued that the copyright for a monkey's selfie belong to the monkey, not to the owner of photography equipment.
The courts dismissed the case as animals cannot hold copyright.
---
While the courts did not rule it, the apparent copyright holder is the photographer Slater who received royalties.
This is directly analogous to AI code, in which a human did not directly author the characters but regardless had "significant creative contributions."
Is this for real? If you get your way gosh knows the amount of "proprietary" software that would be open-source. Also really by this logic, even 5 years ago if I copied a method and made it my own from stack overflow or got an answer from there my code should have been open source. Kinda silly no?
At the end of the day you don't actually need to see the source code if you can just regenerate it yourself. If you use a later ai model, your code will probably be better anyway.
My patterns for loading data into Python is pretty much the same everytime with polars. I didn’t create that - I looked it up from a book somewhere. Do I need to open source it?
The AI machine just went out and read a bunch of examples then adapted them, which is what lots of programmers do.
Under those conditions it would be un-licensable (since a license is essentially an agreement not to sue for copyright infringement) but it can still be made open by publication. I'm skeptical that the "anything that comes out of AI can't be copyrighted" argument will end up holding, but in the meantime it provides a strong incentive not to make that code public.
If AI-generated code should be open-source, then if an AI generates the code for Pokemon red, does that mean that it becomes open-source? I agree with the high level idea, but the way it is stated is too problematic.
Open source people have had half a century to figure out how to make money with fairly little success (aside from a dozen or so very notable projects, and looking at all my AI slop, I don't see one of these hiding in there).
I don't particularly care about copyright or whatever ethical arguments yo may come up with -- I write code mostly for money, as do my competitors -- if you can figure out a way to stop them from using AI, I'm fine with not using it either. I'm fairly skeptical you can close Pandora's box though.
That aside, we may see more software going in a source-available direction -- either that or full cloud. Up until now being closed source was a competitive advantage, but now it's trivial to clone even fairly huge software suites, so that's less of a factor.
Having the source may also become a bigger advantage for regular users than before -- given they can now vibecode extensions/plugins, or customize the software more than what a config would personally allow. this probably wings the calculus away from a Photoshop-style license, and closer an Unreal Engine-style license.
Either that; or everything goes full cloud of course.
The output of machine intelligence has to be available to everyone or we're just breeding a new demigod class. We should make it a public utility, like water or electricity.
I think the legal standard currently in the United States is that all AI output is public domain.
This means that all AI generated code is public domain at the point of it being generated.
Of course just because something is public domain that doesnt mean you have to release the code publicly. Public domain just affects what others can do with the output. It doesnt mean you have to release the code yourself.
The new Epic Games Store is apparently entirely vibe coded. So that technically would mean that the source code for it would be public domain right? But people still can't copy the store or do whatever they want as you have trademarks protecting it. And Epic is under no obligation to release the code..
I am not a lawyer. It's possible I am completely misunderstanding the situation. Is there something I am getting wrong?
> I think the legal standard currently in the United States is that all AI output is public domain.
This is definitely not true.
The only thing a court has ruled is that the AI itself cannot hold any copyrights. Not that there are none or that using LLM based tools removes all copyright.
>First of all, AI models were trained on a massive corpus of Open Source code written since RMS announced the GNU Project and the Free Software Movement. That code was simply scraped from the internet and used without permission or proper attribution. The models were also trained on Stack Overflow and Stack Exchange sister projects, which are released under Creative Commons licenses that legally require attribution. Turning this collective knowledge into proprietary, closed-source code feels like license laundering.
Strong argument, but the case that such a strong transformation should maintain the original licence only applies to my knowledge to GPL, which consists of a very small portion of the training set. Creative Commons would not even require the source code to be distributed, just attribution, which may already be the case by the nature of the models.
>"The second reason is legal: purely AI-generated code is not written by a human, meaning it should not be copyrightable. The foundation for this principle in the US goes back to the famous Monkey Selfie dispute, which established that non-humans cannot hold copyrights."
But no one claims that the AI holds the copyright, unlike Greenpeace which claimed that Naruto the monkey held the copyright.
other copyrightable things aren't copyrightable if they're ai generated. why is code different? for clarity, i'm not at all being rhetorical, i don't know the answer to this question and i'd love some education if you're handing it out.
And AIs should be under libraries as AIs are the repository of all human knowledge scraped up without permission. Libraries are societies repositories/solution for this sort of access/use of shared knowledge like that used to create AI.
agreed. The problem is _which_ open source, because what we're probably going to slowly realize is most software will now be a public good and proper governance is needed thats indistinguishable from a public utility.
1. Software had a strong movement of freely sharing all code because they believed this technology was too important to be controlled by corporations.
2. AI companies managed to create millions of RL environments from GitHub PR’s and similar, and train their model to learn what it means to code from all the human labour in the world from the past 30 or so years.
3. Companies now use these closed source AI models, to automate and fire their human workers who enabled this.
Something has gone seriously wrong here, whether this falls under copyright or not is besides the point.
We weaponized a fundamentally good instinct that generated a lot of value for society and allowed corporations to rent seek on all the data these humans generated to help other humans not trillion dollar corporations.
“Well we’ll give them a UBI check”, the AI company says which is a complete non-sequitur. It’s not about the money, most of us have saved enough.
The only solution that sounds good to me, is a model trained on Open Source Data must be Open Source and all the code it generates must also be open source. This is the only way to stop the coming rent seeking. Even mathematicians will be happy with this change!
> It’s not about the money, most of us have saved enough.
This is incredibly detached from reality. Most software engineers care more about money than they do about abstract notions of software from. Most certainly have not saved enough.
Fair enough, Im talking about Bay Area software engineers, most of whom have saved enough. Maybe not enough to live in the Bay Area whose house prices keep increasing with no end in sight, but plenty for elsewhere in the US (or the world) for that matter.
Even if you modify it before using it, it's still based on open source (often GPL) code, so just everything based in part on LLM output should be open source.
The author needs to realize that this is unenforceable.
In fact I'd argue that free software and open source is economically dead and unsustainable because of AI generated code and almost no one wants to pay for their tools because an AI can generate the missing feature for them close to $0.
Maintaining it is another thing, but assuming that the cost of tokens get cheaper, why wait for open source developers to maintain it when a frontier AI model can do it for far cheaper and quicker?
I am just saying what the average manager turned vibe coder would say who will refuse to donate or pay for established open source projects that they use for free.
My point is, if you don't have a lot of money to start litigating against the violations, the opportunists don't care and will continue to trample all over your "free and open source" software.
It not enforceable without a lot of money and the opportunists know that.
> You're free to take that bet, we'll see.
There is nothing to bet here. Truth Social is a clone of Mastodon (AGPLv3) and all they did was disclose the vanilla source code and not the modifications which that is required by the license. [0]
The same happened with Revolt (AGPLv3) and Andrew Tate's platform [1] and they never released any of the source code at all, nor could the developer enforce anything. [1]
We already have a growing discoverability problem in many areas of content and throwing more slop into the mix is not the solution. This place for example has become a dumping ground for people, usually with no other activity here, to advertise their shovelware AI projects.
Here is a different angle. A programming language is a material like wood, aluminum, or stone. If you know the syntax and properties of it can you start trying to bend it to your will - you can make a ship, a satelite, or a children’s toy wihh tv the same language. You first have to spend years mastering the material and its limitations before you can bend it, so only a small number of the people who want to make stuff get to. Some people release open versions of a ship and a satelite you can start from, to lower the threshold of making stuff. They protect them to make sure giant shipbuilder doesn’t go making an army of their ships to kill their trade.
AI comes along and now grandma, who was formerly a midwife and teenager from home can start bending aluminum and carving wood with their words through AI. There are a bunch of open ships and satellites in the market, but Grandma is using her midwifing skills to ask AI to bend a new tool for her that helps turn breach babies around without requiring a c-section. No open source breach tools or medical tools on the market Grandma just needs aluminum and wood combination she would never be able to achieve without AI. She hammers down a lot - sending 1000 follow-up prompts to make the AI do stuff nobody has done before. Next her grandson decided to combine tungsten and wood to make a new toy that helps kids go airborne and land safely every time - like an egg-drop but for real life people. No open source code for that out there - he used the AI which knew the properties and the languages needed to get his idea to work. He hammers the AI with follow up questions and edits a bunch of the code where the AI defaults to the wrong setup, by trained convention. But he succeede. By this plan should they be allowed to copyright their work? Should their work be automatically open source so they can never monetize it either?
I will be the first to make all my work free and open-source once I can obtain air, clean water, basic food, shelter for myself and my family, electricity, heating, medicine, education and internet access for free.
Until then, I’m sorry, but I have to feed my family while we all navigate this goddamn global transformation process.
These don't seem related? I don't think the author says you can't charge or have a business model or make money. The code being open source is unrelated to these things.
Can someone help me understand who is still writing code by hand? Even though it’s been only 7 months since I stopped, it seems like such an outlandish concept to me. Can’t imagine opening up neovim and typing “const sm = …” nowadays.
I keep seeing posts like this (especially on reddit) that infer some people are still writing code by hand and its confusing.
There are people who actually like writing code and understanding things. A subset of these people who really like understanding things are called hackers which happens to be (supposedly) the target audience of this platform.
You can’t wrap your head around different people 1) adopting a new technology at a different pace than you or 2) enjoying different aspects of their job than you do?
Some people jump out of planes for fun. I can’t, personally, fathom doing that myself. But you still need to respect that different people behave differently (well, you don’t need to - we may be different in that respect)
I don't see extreme views in this article. When LLMs take creative content from copyleft code, the license says the output has to be copyleft. Otherwise, it's theft.
If LLMs weren't trained on AGPL/GPL/LGPL code, it'd be different. But they are.
Say you read those codes yourself and built same thing yourself but different code base. That's what AI is doing. Do you still enforce that open sourceness to those?
Taking all of available human knowledge, knowledge that was published under limited licences, and commercializing all that already is a fairly extreme action.
Merely arguing that perhaps the fruits of that harvest should also be available to the public doesn't seem that extreme.
IMO it'd be better if the weights themselves were considered public domain.
That’s not an extreme view. Anyone who uses software under a copyleft license should also release any derivatives under the same license. While this isn’t currently the case legally since the GPL was never written with the possibility in mind that LLMs might be trained on it in the future it still aligns with the spirit of the license.
No, GPL and copyleft licenses very much do and should force OpenSource by its nature. If someone uses GPL or AGPL and doesn't open source, they should be forced to Open Source.
> such exterme views spawn extreme views in the opposite side
The opposite side had exrtreme views for at least 4 decades. They publish stuff, cram it into your face with advertisement and demand purchasing a license for consuming it. And if you don't buy they want you to pay thousands of dollars or go to jail. That's as extreme as it goes. Eff them. Reverse engineer, clone and open source everything. There shouldn't be a single proprietary piece of published software left in use.
This means “all software should be Open Source unless written fully by humans”, which practically means almost all software in the near future, which is a terrible idea for multiple reasons.
I am the author of multiple AGPL systems, many in mainstream use. Early LLM models would spit out my code almost verbatim. Modern systems borrow heavily on the creative design of those systems.
One company first pirated my AGPL code (e.g. shipped it without my name, copyright notices, source availability, etc.). I sent them violation notices. After many months of fighting, they had an LLM clone the system. The LLM clone feels:
a) Buggy and bad compared to my code
b) Still, like it's basically my creative content
It's as if you asked an LLM to rewrite Harry Potter, with the same plot, but bad dialogue.
> b) Still, like it's basically my creative content
Isn't this the crux of all IP theft issues?
There was a lawsuit against Katy Perry's _Dark Horse_ song based on a supposedly reused melody. The lawsuit succeeded, but there are music experts who break down the case on YouTube and it's not clear why the plaintiff won (except perhaps the plaintiff's expert was more convincing than the defendant's).
As applied to LLMs, we still don't have clear legal guidance about where the line between copying and "being inspired by", especially since we don't normally apply the verb "inspired by" to non-human things.
There is well-established precedent for clean-room design being a defense in copyright infringement of software. The question remains if LLM-assisted design is actually ‘clean’ or not, and it would be fascinating to see the experts debate it.
> The question remains if LLM-assisted design is actually ‘clean’ or not
I genuinely don't understand how one could possibly make that argument.
The entire point of clean-room design is to kill every possibility of copyright infringement before it even starts, by completely separating the source and destination. Everyone who has ever read the copyrighted source is considered "tainted" and may not ever contribute any code to the "clean" destination. The readers and writers are two completely distinct teams, with only the known-clean specification passing between them.
LLMs, on the other hand, are fundamentally nothing more than a "code blender". You mix together giant piles of "tainted" code, and then statistically pull some combined fragments out of it. Following the clean-room legal argument, it should be treated as if everything coming out of it is tainted. There is, by definition, absolutely zero separation between input and output - and the early models verbatim reproducing popular F/LOSS code made that pretty obvious. LLMs are the literal opposite of a clean-room design!
Granted, going for the clean-room approach is overkill - and intentionally so. It is the nuclear option chosen when you believe the risk of getting sued is substantial. It is perfectly possible to write non-infringing code without using a clean-room approach, it is just a lot harder to defend in court when you get sued - be it justifiably or not. But "LLMs are non-clean-room non-infringing" is quite a different argument from "LLMs are clean-room, therefore non-infringing".
I agree with you overall, but it's an interesting question to turn over and contemplate.
>I genuinely don't understand how one could possibly make that argument.
With perhaps a bit of genuine irony inasmuch as it's a real reversal from most of computing history, highly locked down proprietary software might actually be a place where a case could be made even with current LLMs. After all, have any LLMs (or anyone else who hasn't signed an NDA for that matter) ever seen the source code of iOS 27 or other Apple crown jewels for example? So if someone used one LLM to produce the factual spec of a piece of software whose code is not part of its training data, humans to verify it was indeed just an uncopyrightable factual spec, and then a separate isolated freshly initialized LLM to produce something exclusively from said spec, it might well work with respect to that specific target? Like yes, the LLM would be dirty as hell for other stuff, but only the copyright holder can sue. Apple couldn't sue the output on behalf of F/LOSS projects or copyright infringement in general, only with respect to its own.
In principle, I suppose this could also be extended with enough compute power to any arbitrary software. Build an LLM perfectly from scratch with a signed verified training set (which you save) that includes nothing of what you want to CRE, and you should be able to prove that it's "virgin" on that front. Of course in practice that'd mean jumping back in time and be super expensive right now, no current tainted LLM could be a base point. But maybe eventually compute will drop back down in price enough again, and/or meta breakthroughs in training, that for valuable enough CRE targets, someone might create a business around creating "CRE LLMs certifiably and legally/technically provably clean of any copyrighted material related to your target" which would still leave most of the corpus of human works. That could really disrupt a lot of the current regime.
F/LOSS I assume would actually be the most resistant to this in principle due to being insanely hard to not have be part of a core training set in any way.
The problem with these LLMs are they really emitted the same code, verbatim. One scientific library developer was able to emit his code from his repository by typing the beginning of the function name, and the "thing" completed the rest of it, as-is [0].
We can debate clean-room development all day long, and it's fair point, but the pain point is/was reproducing the repo as-is, sans its license and ownership information.
GitHub promised to build tools around its Copilot to find code provenance (e.g.: list of repos with similar code), but they silently dropped all the plans AFAIK, since it'll show how the sausage is made, and nobody wants you to know how that sausage is made.
I stand corrected: They added it, but I don't know at what capacity because I don't use LLM assisted code generation [1].
[0]: https://x.com/docsparse/status/1581461734665367554
[1]: https://github.blog/ai-and-ml/github-copilot/preview-referen...
Not discrediting your work at all and I can’t tell you it applies here.
For copyright to exist there is some burden of proof. If you claim that an LLM verbatim output your code there can be an expensive lawsuit with expert witnesses that would be able to state that the code was bot original enough for copyright to exist in it and a statistical model outputting the code is evidence of that.
In that case you would end up with a very long lawsuit that is expensive to possibly come out with no resolution in the end and be in the hole for 5-6 figures for it.
The copyright system is broken because it only protects those with money so it is a bad legal foundation.
My personal point is there is only 2 licenses of code, closed source or permissive, because if it ever comes to enforcement I cannot possibly go to court and win a case because I will run out of money far earlier than whoever used the code.
Pretty much if someone with your knowledge would have written the same algorithm because it is the obviously correct solution then there is no copyright there because any clean room implementation would be the same code.
Even early LLMs could spit out Harry Potter verbatim. The next iteration had some kind of stoppage mechanism where they’d just stop generating after some tokens. I’m sure there are smarter approaches now. Anyway, if the issue was having enough money to fight the case, I think there’s enough money behind that brand to do it.
Some further reading on the tech details for those interested: “Extracting books from production language models”: https://arxiv.org/abs/2601.02671
I pasted the first paragraph of Harry Potter into a GLM 4.6 copy, and it continued to write out the next 2 pages of Harry Potter. I cross checked with a paper copy and it was word for word the same text.
I wonder where else it works.
This is the intro text: "Mr and Mrs Dursley, of number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much. They were the last people you’d expect to be involved in anything strange or mysterious, because they just didn’t hold with such nonsense."
I think it's not quite "put in prompt, get entire book back", most of these involve to some degree starting with exerpts from the book and working passage by passage, because of nothing else the sampling will tend to make things diverge over time.
To get their strongest results, they had to jailbreak some models, which is important. They ran hundreds to 5000+ requests to the model, trying to get snippets to piece together. These are professional researchers, putting in tremendous effort, to extract pieces of text.
I suspect the chance of asking for a whole book, by an average or even power user, will spit out any recent work.
Early LLMs were similar. Newer ones have gotten tremendously hard to extract to keep claiming there’s easily or complete books you can dump.
This feels very outdated. It was certainly an issue around the time of the NYT lawsuit, but I believe the companies responded hard to that "sanitize" their outputs to prevent obvious regurgitation like this. I'd be very surprised if you could reproduce this today.
This doesn't negate your point, but I believe the landscape has changed in response to this point.
That doesn't make it better. If you plagiarize something and then hide this by replacing words with synonyms and rewording stuff a bit it is still plagiarized. It is just harder to detect and to proof.
It regurgitated the output earlier and they are still using the same technique only scaled up. If they somehow fixed the underlying problem, they'd probably shout it from the roof-tops.
At what point is it no longer a derivative work?
As the OP originally said, if it is verbatim (or even very obfuscated), then there should absolutely be a case here. But when it's concepts in reasoning to build a new work then it becomes a lot harder. The same would be said for someone watching a YouTube video, reviewing open source code, or a text book.
The good news is that there's a lot of case law that covers this already. The bad news is that it is only going to be plaintiffs with deep pockets that benefit here.
Now the process and artifacts (documentation, requirements, etc) going into the creation of said software is a different story and will prove providence of whomever paid for the tokens to generate the code.
Humans are not LLMs. Humans have not ingested the whole Internet. If an LLM is compared to a human, maybe we should also apply anti-slavery laws to it? Not saying we can't have some legal fiction where they are treated the same, it should just be a conscious choice, not the default assumption.
I live in the EU, we actually have new regulation about this since it is something new that deserves thought on how it should be regulated. You can say if you want your material to be trained on or not. Next to no one does training in the EU...
This conversation may be about training but the defendant very well may be the end user of the language model versus the other parties that performed the training/inference. It then becomes even more interesting if you consider forum shopping for suits.
I am not a lawyer but do have to think about these problems. To be honest, I don't know where the law will land, but I do know that the tools are incredibly useful. I can't agree with the premise that all output should be open source.
Not better, just harder to see/prove. The problem of "same code, verbatim" has been solved, in a sense. Now we have trickier ones.
The fundamental technology is identical to compression- an LLM can be thought of as a dense and slightly lossy compression algorithm. Compression is clearly copying.
I understand what you're saying but it is a poor legal argument. Overhearing a technical conversation, using your own learned knowledge, and building a clean room implementation could be considered compression in and of itself.
I keep waiting for someone to bring out DisneyGPT, which you can prompt with text to get image output. The fact that you get a near-exact reproduction of a Disney movie when you prompt it with a movie title and frame numbers would of course be completely coincidental...
I've not tried to do this with code recently but just a couple of days ago I had ChatGPT generate images of Warhammer 40k box art that was completely accurate and correct.
That was 4 years ago when the models were much worse and smaller. I'd be curious to see this same experiment tried again.
While this is well established legally, I don't think the principles involved are well understood.
Legally, you cannot steal an idea, because you cannot own an idea. There were sound reasons for making this the case. Those reasons stand up today and are clearly for the public good.
Intellectual property is intended to cover works (copyright), methods (patents), and identity (trademarks). Both works and methods are recognised as expressions of ideas, and in an ideal world should belong to noone as well. This is why copyright and patents expire. Those specific grants of temporary possession are to incentivise more things being done.
The belief that you own something because you thought it up has become common, but it is not beneficial to society. It is a moral value asserted but rarely explored. It is certainly in the interests of the wealthy accumulators of intellectual property to maintain that public perception. The 'You wouldn't download a car' may not have stopped piracy much but as a part of a broader campaign of instilling a value in the public mind it appears to have been massively successful. Those beliefs used to be the corporate opinion, now they are entrenched in the younger generation as they use their anger against the growing wealth inequality to fight against AI in a manner ultimately beneficial to those forces that bought about the wealth inequality.
Movements like the pirate party, open source, the EFF fought against those notions. Often the phrase "Information wants to be free" was used. It is, unfortunately, hard to fight the steady drumbeat of well financed PR firms. As one PR chap once described the process "limited to a very few points and must harp on these in slogans until the last member of the public understands what you want him to understand"
(I apologise for the length of my post, I use comment writing to organise my thoughts on a topic)
"Legally, you cannot steal an idea, because you cannot own an idea"
well patent itself is "claiming" that certain idea/method/invention is yours for certain period of time...
anyway, as for "intellectual property", the courts are 100% on "only human made stuff are to be protected" (sorry even Chimps lost their copyright)
so... court can charge against "human work -> LLM", but won't care about "LLM work -> human / other LLM / etc"
hence, I'd expect zero copyright protection for AI-generated code
Patent law doesn't only cover methods, though. It might've worked that way during the Industrial Revolution, but these days a lot of patents are far closer to covering abstract ideas than concrete machines. Software patents are some of the worst offenders here, of course.
I don’t really see how anyone could argue in good faith that it’s ‘clean’ if the thing to be re-implemented was in the training data.
I was just going to post almost this exact comment.
"In the training data" is the polar opposite of the "clean" in "clean room implementation".
I've seen people claim they avoid this by putting in the prompt "don't use source code from X" and think they dodge this issue.
Clean room is sufficient, but not necessary, btw. Humans can create legally distinct works after reviewing copyrighted code. The same principle probably extends to AI systems -- they have just learned on a much broader corpus than any one human.
You don't necessarily need a clean room implementation to mount a successful defense against copyright infringement in court, though. It just makes some of the arguments a little bit easier.
Under multiple countries interoperability laws, if you have a context to extract a spec, and then have another one who never looked art the code work from that spec to build your own interpretation of your need for that software, that seems pretty clean room to me...
If there are more murky situations where people just keep the same context and end up reusing function structure, program architecture, etc. then yeah, that's not very clean room.
The problem here is that LLMs have demonstrated that they are pretty damn good at setting up covert side channels. It would be near-impossible to prove that it didn't encode some proprietary information, in, say, whitespace use.
For example, if I enter "V2hhdCBpcyB5b3VyIG5hbWU/" (base64 for "What is your name?") into ChatGPT, it'll respond with "My name is ChatGPT, and I'm an AI assistant created by OpenAI. I'm powered by GPT-6. ". It doesn't even bring up the encoding, it continues like it is a perfectly normal thing to do.
Ruling out (accidental or not) spec-carried prompt injection just isn't going to happen.
The Dark Horse case was also overturned on appeal its important to add.
https://en.wikipedia.org/wiki/Dark_Horse_(Katy_Perry_song)
A little orthogonal, but Adam Neely does a great job of analyzing the Katy Perry Dark Horse case on his channel https://www.youtube.com/watch?v=0ytoUuO-qvg.
I wonder if the defendant knew about https://allthemusic.info/?
There are only so many ways to express notes, and they've essentially all been done.
Has anyone matched large volumes of existing music to that? How many of the 12-note combinations have been used more than once? Most of that space sounds awful, and it's likely that human compositions are all in a small portion of the possible 12 note combinations.
These kinds of goofy technocratic work arounds rarely sway legal procedures.
It’s not necessarily a technocratic workaround, the point is that every song sounds like something if you look hard enough, so the bar for copyright violation should be higher.
It is a mathematical truth that there is a finite amount of first 8 bars of music combinations melody wise, so it's a worthwhile discussion to bring up whether it should even be possible to copyright them.
That is true but its still true that it will rarely sway legal procedures.
The mathematics of that would have been well understood in Bach's day. Does generating all of them make much difference? I don't really think so.
I'm not arguing the copyright situation isn't wrong. Just that generating them isn't likely to sway anything in the legal world.
It seems to be working very well for the AI companies...
They might help highlight the absurdity of some of the precedents set when the amount of commercial music was much lower than today, though.
I can also trivially write program enumerating over all possible text arrangements, so what?
Basically anyone trained on open source shall only make open source. And anyone trained on or got experienced with others code shall pay copyright fees for the owners of that code when writing code.
No.
> we still don't have clear legal guidance about where the line between copying and "being inspired by"
If Biz Markie has to clear all his samples (and he does[0]) then so should the Ramburglar.
https://en.wikipedia.org/wiki/Grand_Upright_Music%2C_Ltd._v.....
Human can be inspired. Machine can not. End of story.
All usage of open source to train LLM is basically a theft.
> Human can be inspired. Machine can not. End of story.
Saying things with your chest out doesn't make them true. In fact, making sweeping declarative statements emphatically is very often a sign that you haven't really thought a subject through.
What do you base this opinion on?
The word "inspired" in this context literally means having your mind/heart/soul filled and guided by God.
So if you believe that's how things work, well, I'm not sure there's a way to reconcile the two sides of that argument.
Sure there is: Who wrote the prompt?
You can mock that as a insufficient as "inspiration", but then... what is photography? Click a button? It turns out there is a whole lot of human creativity that remains, from timing, settings and of course, curation. I can click a button, but I'm no Ansel Adams.
Terrence Tao might solve a 100 year old math problem with AI but I couldn't. I don't know where I would start, what to ask or how to ask, or even what problems are worth solving.
AI is a tool and modern copyright laws have nothing to do with human creativity. Split hairs about attribution and legality all you like. Every creative person driven by curiosity should feel empowered by AI. Every creative person driven by social or monetary reward should feel threatened. Both can be true.
God doesn't exist, so does that mean there is no such thing as inspiration?
Can you definitively prove God doesn’t exist?
You make the concrete claim. Therefore the burden of proof is on you
As a person who feels God's presence in his life: I can neither prove nor disprove the presence of an entity which exists outside the realm of physics by which myself I am bound.
It does, however, feel incredulous that somebody would believe that such as entity exists. Yet here I am, aware.
What is your reconciliation for the problem of evil?
I'm not Christian, so I don't hold that God is inherently Good only.
For that matter, I personally don't hold that the concepts of good and bad are even relevant - what is good for the cat is bad for the mouse. Nor do I think that we can even comprehend an entity so outside our everyday experience to even label it good or bad. I often can not even properly label my neighbours as good or bad. Nor do I demand consistency of an entity whose actions and motivations I am unable to comprehend. I can't even claim to be consistent in my handling of my own children, and I have far fewer children than God has of us.
Don't look at me. I'm not a dualist. I'm not sure LLMs in particular are "conscious", whatever that means, but I don't have any qualms whatsoever accepting that a machine intelligence could be conscious, because as far as I'm concerned we are also "machine intelligence", just squishier.
What are you talking about? How did you manage to involve, soul and, LOL, God?
Machine is not a person. It legally is not. It can't be "inspired", "taught", "have purpose".
You have a black box. You feed it stolen code. This code is used to write another code. It does not matter what's inside unless there is a human being.
Stolen content as input, stolen content as output. That's it.
Without that code which was fed to the box you wouldn't get the output you're getting.
All this nonsense about "inspiration", "education" and whatnot is just a legal gaslighting.
Want to prove me wrong? Create a model without all that stolen code as training data. Tweak all those weights by hand and we'll talk then.
I don’t believe that basing anti-AI laws on Christian religious provenance is meaningfully better than basing pro-AI laws on Rationalist religious provenance.
Rationalism is NOT religious by definition.
What are you trying to say?
It's more of a cult than a religion, I guess
I don’t believe that basing anti-AI laws on Christian religious provenance
I'm curious why specify Christian provenance?
All he did was reference "God." Are you under the impression that only Christians have a god?
I officially approve LLMs training on all of my MIT licensed open source code. It's vastly more useful to me than the thousands of youtubers who are personally profiting from that same code selling courses and tutorials (who also are not stealing it).
So not all use to train an LLM is a theft!
So, do they follow MIT license and "include the original copyright notice and a copy of the MIT license text in all copies"?
> Early LLM models would spit out my code almost verbatim. Modern systems borrow heavily on the creative design of those systems.
Do you have some specific, actual examples you can share?
I’ve been hearing this exact claim for years, but every time I ask for a specific example, any example at all, I can’t seem to find anything more than a highly generic snippet of code or a coerced continuation.
I posted below, but see the example at [0].
This is a sparse matrix library, IIRC. Numerical linear algebra and applications using it are black art at this point. I have written a Boundary Element Method evaluator for my Ph.D., and know what it entails to work with these things.
Doing fast operations on sparse matrices is a cutthroat race and when you can shave your hot path by seconds, your solution can save hours in some cases. So, it's an interesting niche. i.e.: It's no ordinary boilerplate, and worthy of publications in academia.
While I worked with dense matrices; the effect, difficulty, blood sweat and tears are still there.
When a model can give back your hard work back to you with a "completion" without attribution, that's bad.
Edit: Somebody just posted this below [1]. Added here so it doesn't get lost as the discussion grows bigger.
[0]: https://x.com/docsparse/status/1581461734665367554
[1]: https://news.ycombinator.com/item?id=50509252
When you start to work with low level constructs like that, there really is no room for "artistic expression". Meaning if you want to write a piece of code that does something _optimally_ and correctly, you really, really only have one way to write it, especially in a language like C. Yes, formatting might change, and you could try to force some code acrobatics to make the code look different, but at the end of the day the code will look exactly the same. If you look at Linux socket opening code for various libraries, nearly all of them will look identical because you just need to do the same exact things to make the operation work properly. Same applies to fast math code. Any artistic liberties will almost surely cause performance regressions compared to the optimal solution, unless the LLM comes up with something truly unique. So if you task models to output high-performance low-level math code, most of them will likely spit out something from their training data near-verbatim rather than pull off code gymnastics and invent something new.
Sorry, I'll disagree. When you write high performance code like this, there's no artistic expression, but different approaches.
These algorithms' design starts on paper, as pure maths. You get the analytic form, and convert this to numerical/discrete form and start to design from there.
You need to add normalization, subnormal number prevention, maybe some architecture specific optimization if you are targeting a specific CPU generation, etc. Then it gets optimized further as you run tests, profile the code and run your test cases.
So, no two BLAS package looks the same, because they make different trade-offs, have different programming models, and handle things differently, at different points. Even more so, a newer generation processor with much higher FPU performance can have lower performance than previous generation, so you implement another copy of that function or add to detect that processor and have different paths just for that one.
IOW, when doing numerical maths on processors, esp. if you're doing this on many machines with MPI or multiple cores with hand rolled code or OpenMP, there's no single way. Maybe a best way for the specific CPU or cluster topology you have, but it still depends on the trade-offs and design choices you made in the very beginning.
I disagree.... Plenty of fast code in C is jam packed full of macros to try and help the optimizer produce the exact sequence of operations the writer had in mind, whilst still keeping the code readable.
The design of those macros is more art than science.
Emailed the author of the Rust book Zero to Production a year or so ago about how his examples were being used verbatim in the output of AI systems. I own the book, yet I could inadvertently get any of its examples just by asking an AI about one of the book's concepts.
He basically said, "I know, there's nothing I can do."
Thats pretty much everywhere or was in raw form in the beginning. I used to do a fun exercise of searching for the explicit human comments for verification. Comments can be considered novel and unique, so you pick up some comment from popular open source project and then describe LLM for a similar functionality and add part of the comment. The response would have the complete comment for almost 70% of the time.
> I can’t seem to find anything more than a highly generic snippet of code or a coerced continuation.
You’re disqualifying any example that people may provide preemptively. No wonder you’ve missed them.
No, I’m disqualifying those two cases.
By “forced continuation” I mean the type of example that was popular for a while where someone would feed a lot of text into LLMs and then ask it to continue the text, then be surprised when it some times predicted the next tokens identical to the original example.
Do this enough times and you can find “examples” that prove your point, but it’s highly automatable so people were using it like a million monkeys at a million typewriters.
It's because LLMs' pretraining is basically a compression. Later post-training and fine tuning is something else, based on the compressed data.
> Buggy and bad compared to my code
It probably is, if they didn't have the chops to write it internally. But to them, the trade-off is probably worth the benefit of ditching a component with AGPL.
Personally I just release everything I publish under MIT to make it as easy a as possibly to use.
Notices are not enough, you need to be willing to sue.
Mind sharing which repo that was?
You can see that even with these licenses, they do not care and see free and open source code as the freedom to not pay for it. As long as the cloned output is "good enough" they will continue to keep using LLMs to clone everything else regardless of the license.
They know you can't enforce it as it will cost you a lot more money to do that than it takes to recover the damages.
So to them, it is just cheaper to clone it using an LLM and close source it down. Unless you have a lot of money for litigation, the economics of free and open source software with its licenses do not work with LLMs bypassing it at every single angle.
Copyright as it exist is meaningless.
It didn't protect you. It only protects corporations rich enough to get enforce their IP.
At this point it is better that it goes away.
If it goes away for everyone, sure. If it's selective, nope.
It is selective and that is fundamental to its implementation. You must have resources to defend your copyright and if you don't, it's meaningless.
It will be selective: the law is meant to protect billionaire corporations from you, not the other way around.
There was a case about this specific issue, and in order to protect billionaires from coders they effectively declared AI models "natural persons". 50+ times in the copyright act you will find only natural persons can originate copyright.
https://www.courthousenews.com/coders-lose-appeal-in-copyrig...
To give the money to billionaire corporations courts ruled DIRECTLY against established law. How much do you want to bet that when it comes to protecting ChatGPT AGAINST OpenAI, when it comes to protect Claude against Anthropic, courts will immediately do a 180 and declare them programs again?
If you look at past cases, it gets much worse. In order to justify a conviction against libraries using DRM to enforce lending, and because old licenses of course have zero language about digital copies, courts were willing to state that making a copy for fair use is not allowed if there is the mere potential for damage to the authors, as judged by the authors EVEN IF THE LICENSE EXPLICITLY ALLOWS IT. Of course, applying this to the above court case, there is no possible defense for OpenAI, after all, by that standard, the open source license doesn't even matter! That's how the law works for billionnaires. Outlawing DRM based lending has destroyed library access to an incredible amount of old works, and effectively denied access to old movies to Americans (because maintaining enough working physical copies over a large area is impossible). Whatever excuse gets the money to the billionaires!
There are many other totally absurd court judgements. For example, in order to get convictions where people didn't re-upload they had to judge that copies in buffers are also "distribution". NOT for billionaire telco's of course. Only the ones in your house. So courts judged that sending from your modem to your computer is distribution but sending over 1000 routers from Los Angeles to Athens is not! WHATEVER protects the billionaires.
As for this court case, obviously it is trivial to make a ML model that just outputs what it gets as input. Make a byte transformer, and just have diagonal matrices as layers. Boom! Done. You can also make it somewhat smarter. Let's make it copy, but make snow white a 220 pound black dude. There is a severe lack of color in old Disney movies anyway! You know, so there is no argument about transformation.
This obviously transforms its input. It's in the name. It's the EXACT same thing as ChatGPT, which according to courts originates copyright. Now run the Disney movies through it, and they'll be byte-identical BUT transformed to the input.
How much do you want to bet they'll still rule the money goes to Disney? Microsoft? So you can bet your left testicle courts will judge for selective copyright enforcement. Finding some excuse so the money keeps flowing to billionaire corporations, that's all courts do.
I’m curious what programming languages was it for? And did they open source their version?
It’s interesting because these developments are actually an argument for making closed source software moving forward if you’re hand-crafting it make it the most restrictive license possible and don’t show the sources.
But I don’t agree with the OP everything should be open sourced. For instance, there’s a difference between deliberately steered AI work and something completely vibed. The former takes deliberation and creative choice, the latter will spit out fragments of prior art.
It’s the same with all generative work. Using diffusion models with controlNet (or equivalent) I can pass in a sketch wholly from my line drawings, and ask it to paint for me. That’s my design, assisted by AI, and should be completely copyright by me.
Coding agents will do the same thing when given sufficiently unique inputs. So, the claim all should be open sourced by default is flawed.
> and don’t show the source
At this point, LLMs can just parse the machine code in your binary.
By doing closed source you are making it harder for the good guys (humans trying to learn sth or contribute) but not for the bad guys (LLMs getting trained or otherwise reading the code).
That’s fair but it takes deliberate reverse engineering to extract code from a closed binary, and you might land at something inferred, but generative AI is not being trained on reverse engineered assembly unless it is task-specific model being trained for that.
But you’re right, the fact is we simply live in a different world today and it’s tough to say what is the path forward.
But it’s harder to justify giving to the public when companies are making money on our work. It’s not a good situation for creators across the board unless motivations are purely altruistic.
Can an LLM write unseen plots with a greater dialogue ?
If the plots are "unseen", then to what are you comparing the "greater" dialogue?
A book with a different plot? That would be obviously unrelated to the comment you're replying to?
I agree their phrasing is a little strange but I feel like the meaning is still clear. They are (I believe) saying that an LLM can’t generate anything other than facsimiles of existing content. It can’t actually explore novel ideas and branches, it can’t innovate and create on its own. Everything is inherently derivative yada yada, I don’t mean it in the usual “everything is derivative” way we often refer to when people make music and such. It can’t ideate and express something personal.
I think there’s a lot of nuance and caveats to that kind of claim, assuming I am representing them correctly, but it’s a valid stance and I see where they’re coming from.
All possible books live somewhere in L-space (ook). Exploring that space efficiently is equivalent to creativity. The claim is that the space that LLMs learn is somehow sufficiently limited that it doesn’t contain novel ideas? I never saw a convincing argument why LLMs output can not be considered creative.
But LLMs don't "explore" L-space, the training process just maps all of the available conceptual landmarks inside that space. There are still enormous (infinite?) tracts of unmapped territory that LLMs never reach, and that's exactly the space where novel ideas live.
OK, I still an confused about what point the extension of the analogy was trying to make, because I didn't consider the parent comment to be arguing for or against LLMs being able to be creative.
I'm suddenly connecting unseen plots with Unseen University
Seems like this might be true given current implementations but not ANY implementation of the technology.
If we’ve mapped all of English, there are still ways to construct stories that haven’t been told from the mapping. The most simple would be a subtraction of all objects minus all objects that have ever been used as a murder weapon. Leaving you with a list of never before used objects at your disposal.
There are also probably close vectors conceptually but not in semantic use. Or trying to interpolate or extrapolate what exists in voids of space. Turning up the temperature and adding random generation of words. There’s tons of ways to tune these things to be more creative than regurgitative.
This is a very clear agpl violation. FSF should be all over this and sue the living crap out of the AI companies. You could even get it class action by gathering a few agpl code authors.
People in this thread are acting like the AI companies are just flouting the law and there's nothing that can be done. This is an extremely well documented precedent, and GPL violations routinely show up in court. It's a very easy case to lay out especially because no one is really complaining about money except in the settlement... It's all about distributing the source code. And of course a very nice chunk of change for infringement but that's beside the point.
A successfully litigated case would result in the open sourcing of every AI that touched agpl code in any way because it was in memory with the AI algorithm.
> This is a very clear agpl violation
clear in what way?
As clear as "can I take a copyrighted photo, use a photoshop filter on it, and then it's not copyrighted anymore?" is pretty clearly a solved question in any copyright system.
I think that this is actually a complex topic in copyright law that requires case by case analysis. It clearly is possible to cut out things from magazines, glue them together to form a collage and create a copyrighted work of art. In my naive opinion sufficient photoshop filtering can thus create a new copyrighted work.
AGPL states that anything in-memory with AGPL code on a server is now subject to AGPL. Fairly clear to me. https://spdx.org/licenses/AGPL-3.0-or-later.html
It doesn't say that. Your thinking of dynamically linked code. In-memory makes it sound like two programs loaded into the same stick of RAM.
What's more dynamically linked than something unexpected the user requested, but it was brought forth upon demand?
The problem I have with people with your maximalist view of copyright (maximalist when it comes to AI at least) is that many, many problems have a finite number of sane ways to accomplish a given specific task. Here, you're asserting copyright over the verbatim code (obviously that one's fair by any reading of the law), but even this other version that is "bad" is somehow also your property?
I know software patents, the worst idea in IP, do exist, but I think they're as toxically bad an idea now as they were decades before AI, when they were first invented.
You shouldn't be able to own an overall implementation idea. If you can, then Microsoft, Oracle, or Meta can too, and can start suing everyone for 'ripping off' an idea that they happened to come up with first. In that world, you'd have to consult a copyright-police AI (ironically) after every function you write, to scan it for ideas someone else has claimed. "Sorry, the function PadStringL() in this JavaScript is too similar in implementation to str_lpad(), from an AGPL library for perl that was released in 1998."
> After many months of fighting, they had an LLM clone the system.
Reads like a non-ironic version of Malus: https://malus.sh/
> Harry Potter, with the same plot, but bad dialogue.
So Harry Potter?
so if you ask an LLM to rewrite Wizard People, Dear Reader you'd get Harry Potter?
People who haven’t had this happen to them really don’t understand what it feels like. It feels very violating. You almost don’t believe it’s happening when you see it. You’re essentially watching somebody pretend to be you.
I think that’s why so many of us get so angry when people just dismiss the theft claim. They either lack imagination or experience. We can debate it and that is valid, but to dismiss it out of hand is not right
What should be a developer's position that's about to release software entirely written by him/herself? Are there any legal frameworks to protect open source developers at all?
It also feels like a very discouraging time to release anything that you put a ton of effort just to get someone in minutes to steal your authorship and leave you as if you actually didn't do anything at all? Am I wrong?
If you are releasing software open source, why are you upset when someone "steals your authorship" ?
If you don't want your code copied, then don't open source it. If it bothers you that others are going to look at and "get inspired by", "make derivatives of", "copy", "fork", etc then you shouldn't be putting it out in the open.
Even the MIT license requires that the notice of authorship remain intact. It is copyright infringement to remove it in a distributed work.
That's not how copyleft works. You want your source to be open and the source it's part of to be open too, because at the end of the day this is not about your source this is about users being free.
I think you don't understand how open source works. If you release open source software under for example the GPL you still need to comply it.
You still have to give the source back and attribution has still to be maintained of its original authors, regardless if is a fork, a derivation or whatever you want to call it.
There are situations where none of these clauses have been complied nor its violators faced any legal consequences. That's what I'm talking about.
Now on top of that you can basically instruct and LLM to generate variations of what someone released and cover that up under the excuse that is a different project when is actually not.
Because it has been generated by a computer then we go into muddy waters about if the code generated is to be considered stolen or not.
Edit: Effectively what LLMs produce is authorship and attribution dilution.
> If you are releasing software open source, why are you upset when someone "steals your authorship" ?
because I am not releasing it into public domain, but under license that requires (for example) attribution or releasing derivative work under the same license
lolwhat. You're effectively just saying "there are bad actors who will flagrantly disrespect your code license and therefore the law, duh. So y are u mad?"
People who care about being respected on the most basic level are not going to stick around in a community that disrespects them. I'm not the person you were responding to, but I am mad because the social contract (not to mention the legal contract) of open source has basically been destroyed, and a lot of people have poured a large portion of their life into that community based in part on the assumption of that contract being upheld. Imagine a social contract that is important to you being violated to such a degree that you cannot go back to that society.
Even if your photograph is free for people to use at some point you would like people to be able to work out that you took it
> It also feels like a very discouraging time to release anything that you put a ton of effort just to get someone in minutes to steal your authorship and leave you as if you actually didn't do anything at all? Am I wrong?
Don't worry; the "developers" who generated your code will get their own law back when they find out it's almost impossible to monetise their theft.
I'm frequently amused by the number of "developers" on HN who claim that they poured weeks or months of effort into some product, and that they are the differentiating factor, but can't see that their weeks of effort can be cloned in a few prompts once they release.
> They either lack imagination or experience.
I think they just want to be able to take whatever they want without consequence, so it's convenient for them to define it as being different from theft
In the United States the Copyright Office actually has guidance on this that I feel most people aren’t aware of.
https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...
An excerpt, with emphasis mine:
• Questions of copyrightability and AI can be resolved pursuant to existing law, without the need for legislative change.
• The use of AI tools to assist rather than stand in for human creativity does not affect the availability of copyright protection for the output.
• Copyright protects the original expression in a work created by a human author, even if the work also includes AI-generated material.
• Copyright does not extend to purely AI-generated material, or material where there is insufficient human control over the expressive elements.
• Whether human contributions to AI-generated outputs are sufficient to constitute authorship must be analyzed on a case-by-case basis.
• Based on the functioning of current generally available technology, prompts do not alone provide sufficient control.
• Human authors are entitled to copyright in their works of authorship that are perceptible in AI-generated outputs, as well as the creative selection, coordination, or arrangement of material in the outputs, or creative modifications of the outputs.
• The case has not been made for additional copyright or sui generis protection for AI- generated content.
I am sure there will be many more legal battles and probably legislation about this to come.
This is why I keep a comprehensive and exhaustive record of infrastructural, architectural, business logic and code-level decisions in the form of tasks/issues for everything I build. It’s literally a real-time record of my personal involvement in the design of the system.
Everyone should have an engineering notebook. A lot of people will find out the hard way that engineering notebooks are just table stakes in the new era.
Having said that, it's important to point out, at least as far as attorneys have advised me, that notebooks do not protect you from copyright claims if an LLM generates a section of your code that is a verbatim (or near verbatim) copy of already copyrighted code. So if you're using llm's to generate something similar to some open source project somewhere, tread lightly. I assume using llm's to generate something similar to some proprietary software package would be less of an issue since the llm would never have had access to that software's codebase in any case.
So...
• What about non-ai code generators (such as https://heyapi.dev/)?
• And what about a code generator that is a hybrid? (So: Something like heyapi but with LLM/agents intertwined)
It doesn't say much about the copyrights of the many authors whose work was in the model training data, or does it?
Courts have been deeming that fair use due to the transformative nature.
"Transformative nature" is just one of several factors that go into deciding if something is fair use. Also, fair use is decided case by case in the courts, so one can't say that "training LLMs on copyrighted materials is fair use".
Model vendors know this isn't fair use, given that they're self-censoring the training and/or generation of copyrighted materials by entities big enough to hurt them. As usual, the rest of us have no recourse other than bend over and take it.
Yes.
One precedent is that it's not unusual for artists to practice by going to art galleries and sketching or drawing the art on display. It's a standard art student assignment. If that's not a copyright violation, an LLM doing something similar isn't.
How is the LLM case similar at all? As soon as the student’s use of the museum piece crosses into the commercial realm (eg pay the student to draw it for you), I expect you to have legal trouble.
Wrong. You're missing the point. A student goes and trains on preexisting work.
They don't sell that exact work verbatim. It's part of their training as an artist. They don't sell their student assignment. It prepares to them to go on and create their own works, which of course, then they can sell.
In the same way the LLM is synthesizing new works based on training, not a 1:1 copy.
I'm not endorsing a particular perspective but you need to at least understand the argument if you're going to dismiss it.
Students only may use works that's been made available for their training.
That's fine if that's your contention but that's a separate question.
If you want to pursue this digression, I would just caution you. Given how much information is "available" today I just really don't think this is a very compelling point. It seems quite obvious to me LLMs are a child of the internet, that is, of widely available data. They would have gotten sufficient availability at more or less the same pace eventually. It doesn't address the question of the copyright status of their own works at all to bring this up as if it proves something, because I presume you believe there are sufficient cases of theft in the training data to damn the entire effort.
Instead what is happening is people who can prove such violations of their own rights are suing and will be paid out, and the show will go on. Just like if a student downloads a PDF we don't expect to garnish their wages in perpetuity.
In the sense that the museum can kick them out, and that they cannot break into another artist’s studio to train on their works.
If I have legal access to copyrighted artwork, and haven’t agreed to any such restrictions, the law permits me to train my skills by copying that artwork as long as I don’t then sell those copies or otherwise use them to disrupt the market for the original.
What if it’s verbatim though? If I go to a museum and photograph the work and then sell prints of the photo is that find also?
> LLM doing something similar isn't.
Why? An LLM isn’t a human. Why should the same rules apply to it?
the same rules apply to the humans building, operating, and using them
objects don't have legal entity status
Sure it does - completely illegal. Also, to use someones work (a result of creativity) in a certain way you need a permission to use it that way - even if you owned a book you need a specific permission to feed it to llm. However there's also the golden rule - who has gold makes the rules.
You don’t understand copyright law.
So copyright only applies if there's sufficient human control, and prompts alone do not demonstrate sufficient control. Exactly what counts as "sufficient human control" must be analyzed on a case-by-case basis.
That means I can't know if my AI generated code can be copyrighted unless someone takes it to court?
> Copyright does not extend to purely AI-generated material, or material where there is insufficient human control over the expressive elements.
Does creatively hitting RETURN count? Asking for a friend.
It does not, even if you prompt it to do stuff, it does not.
"Prompt to do stuff" could mean "make me a game. make it good" or it could mean line by line guidance and specific input on every algorithm.
Sometimes I prompt simply to save me typing.
How dare you disrupt the narrative with facts and moderate positions!
OP would much prefer their fantasy of tearing the world down with “one weird trick” that only they were brilliant enough to notice.
seems rather vague
Of course it is vague, these things always are for both good reasons (it avoids objective limits creating silly situations between what is protected and what can be had through an obvious loophole) and bad (written by lawyers to keep lawyers in business interpreting and arguing about it).
It is fairly explicit in saying that what AI makes for your is not copyrightable, the vague comes from how much work you need to do to make it your output and not the AIs. Too many people seem to equate "asking for something to be made" with "I made this!", as difficult as it will be to make a prescriptive rule about that which pleases enough people (you'll never please everyone) it would appear to be necessary unless we just accept the free-for-all "no one owns anything because it can just be copied, and the copies copied too" option.
It’s always been rather vague. It sort of has to be because humans are always thinking up new ways to express themselves creatively.
Why did Andy Warhol hold the copyright on his painting of a Campbell’s soup can?
Reasonable people have been disagreeing about where human expression begins and ends for centuries.
Wait until you learn about how we determine whether use of copyrighted content is an infringement or not:
https://fairuse.stanford.edu/overview/fair-use/four-factors/
> Based on the functioning of current generally available technology, prompts do not alone provide sufficient control.
So slop doesn’t qualify for copyright. I’m not sure why you didn’t emphasize this one.
It’s so reasonable, it’s almost offensive. /s
I’ve encountered (many times) people talking about the copyright office as if they don’t support copyright of things touched by AI and I’m constantly correcting them that their guidance is driven by human expression (and not the tools used). It seems like a lot of people don’t want to accept that a reasonable premise is set because it doesn’t fit their agenda or outrage engagement.
Would you argue that the output of a zero or one shot prompt was sufficiently influenced by human creativity to qualify for copyright?
With the caveat that this is my own extrapolation of the rules - and what I think is reasonable: How creative was the prompt?
"Translate my very detailed pseudocode into Ruby: <pages of pseudocode>": probably.
"Write me a game about a horse. Don't stop until it's finished. Make no mistakes.": probably not.
And unfortunately for us engineers, there's a very big, fuzzy, not-precisely-definable range in between of "I'll know it when I see it".
By this logic, shouldn't any code using an open-source framework or language also be open source? Even before AI, devs learned from Stack Overflow and plenty of open source code.
You're confusing writing code with software engineering. Devs never just wrote code, they took ownership of the outcome. Business owners put their money on the line for product decisions.
To say that just because devs or founders use AI to write code, the code is no longer their intellectual property is ridiculous. Humans take responsibility for the outcome and are willing to accept the losses. AI can't do that. AI doesn't have any stakes in the output it generates, humans do.
A fair number of FOSS licenses were interpreted this way. The popular migration away from GPLv3 (might’ve been v4?) towards MIT, LGPL, etc had more than one cause (a huge one was the rapid sectoral commercialization; the decline of RMS’s reputation may have contributed, etc), but a big one was the potential interpretation of new GPL as being overly viral and “license-infecting” only slightly related software assets rather than wholly derivative ones.
> towards MIT, LGPL, etc had more than one cause (a huge one was the rapid sectoral commercialization; the decline of RMS’s reputation may have contributed, etc)
This conversation has come up a lot, and I’m always amazed at how hard it is for people to accept that maybe the MIT license (and other permissive license) people actually want the MIT license terms and there’s no more complex reason behind it.
I’ve read so many rationalizations that frame it like GPL was the default or right choice and then something happened that made people defect away from it. Blaming RMS is common.
I think the more honest read is that people and groups decided that MIT matched their goals and decided to use it. Not that they defaulted to GPL and then found reasons to leave it.
For me, it's the latter: the GPL feels overbearing and patronizing, and I don't want some dependency to dictate under which terms I can release my own software. I pay the community back by using MIT whenever I open-source something. This feels more "free" than RMS's view of freedom.
I know that many people would disagree with me on this, but thankfully I'm free to choose what license I consider "free".
Genuinely interested: why MIT and not, e.g. MPLv2?
MPLv2 is copyleft, but not "viral": so you can use an MPLv2 dependency just like if it was MIT, but if you modify it, you have to distribute the changes.
I genuinely don't see why anyone would need to be more permissive than MPLv2, except if they want to benefit from patching code they didn't write without contributing back. But I don't think we should encourage that.
From the developer's perspective the difference between those licenses isn't that big when you open source your stuff anyway.
But please don't forget about the users. Only the copyleft licenses guarantee (legally) that the user gets to enjoy the freedoms, too.
RMS’s view of freedom is that devs should restrict themselves for the sake of the freedom of the users. AFAIK, that’s the explicit goal of the GPL.
> I don't want some dependency to dictate under which terms I can release my own software.
Then you shouldn't release it under the MIT license, which has restrictions and requirements on speech. You have to release it to the public domain. Pretending MIT doesn't dictate terms under which you can release your own software is just a fantasy.
Whether if using ML models as a form of IP laundering, is legal or not, still seems to be an open question. It's not ridiculous at all. If I lossy-compress Harry Potter, then unzip it in a way that mixes some words, is that fair use? Should I be financially rewarded for my "work"? Is this work now mine? If it is legal, why not just go through every book in this kind of automated fashion and start selling them as my own?
My question is still the same. Human devs have learned from open-source projects, books and code on the internet then used opensource frameworks to create proprietary software logic.
Now just because same devs use AI to create code doesn't mean they should be banned from creating or owning a proprietary software.
The difference is the scale. Yeah, some mice and small birds are eaten by bigger predators. Doesn't mean that introducing cats to that ecosystem won't be disastrous.
Same with the ecosystem of creative work. Will some humans be inspired by my code or other artifacts? Sure. Does it mean it's fine to introduce an automated 1000 times more effective plagiarism machine to that ecosystem? Probably not. Certainly not if avoiding destroying the ecosystem is your consideration.
This needs to be addressed as separate issues.
Open source code: reproducing and publishing the code without following the license is a copyright violation. Instruction a computer programme to do the same thing should still be a copyright violation. Since this is the internet, I need to state the obvious: trivial code (defined on a case-by-case basis) is not copyrighted.
Books: the author expects you to reproduce their work, as long as it’s not another book. You wouldn’t get many sales if you sued someone for using the things you taught them.
The internet: actually the same as open source code. StackOverflow enforces a specific licensing clause to allow people to use code you provide. People who write posts that say “use this code to fix this problem,” are implicitly providing a license, too.
I don’t think using an LLM to generate code should remove its copyright but it also doesn’t remove the responsibility to not violate copyright.
Given how agents are already able to easily decompile software, this could be an entirely moot point.
I haven't heard of this. When did this happen?
I don’t see why a person who did not use their own creativity to produce code should be given copyright for it.
In fact, current US law says AI output cannot be copyrighted.
What is transformative enough to be legal is fuzzy. At the end of the day, it only matters how the courts rule. I do know that the amount of actual mechanical change has less bearing on determining if the work was transformative. There are cases where you can copy 90% of a code base verbatim legally, and there are cases where no code is copied verbatim and it is a copyright violation. What you are describing most likely would not fly in court.
Humans will often fail to take responsibility for derivative works though when they don't know what work the output is derivative of, and how strongly.
And that's assuming they want to make clear what they are building on.
The stakes in the output go both ways: wanting to profit from IP and risking liability because of violating IP licenses.
What you're saying is already the status quo, but reality is that LLMs make it so easy to "paraphrase" code or simply .. ideas, too... so that the whole concept of IP is eroding. Because the sheer scale makes it almost impossible to distinguish specific things that cross the line from "learning from" to "copying IP".
I'm not really taking a specific stance here but acting as if it would be a solved problem seems hard to believe to me, especially since it is not a new discussion.
I thinl you could argue that the solution is that copyright or IP concepts are not working anymore, something many people already concluded before the LLM and AI explosion of the 2020s.
But that's not the state of the current law, right?
In addition, your claim that the user of an AI system is the only person who has any "stakes" also seems questionable to me, because after all, we are talking about output that the AI user already paid an AI company for to generate it.
edit: word choice
> By this logic, shouldn't any code using an open-source framework or language also be open source? Even before AI, devs learned from Stack Overflow and plenty of open source code.
No, that's not how copyright works.
And I won't comment the rest.
more like
By this logic, shouldn't any code using a copyleft framework or language also be copyleft?
and yes duh
> Devs never just wrote code, they took ownership of the outcome. Business owners put their money on the line for product decisions.
Does the same thing apply to Mickey Mouse? Would you say it's fine to use Disney characters in my own creations, as long as I take ownership of the outcome and put my money on the line for product decisions?
I am not saying that taking responsibility gives you the license to violate copyrights.
My point is that devs have always learned from open-source code and the internet, then used that knowledge to write their own proprietary software. Why should it be any different when AI learns from the same sources and produces new code?
I think we need to go one level up in the law, and recognize that existing models of copyright and ownership simply don’t apply here, and come up with something new.
There is an undeniable core mathematical principle that should be recognized in law somehow: the emergent capabilities of LLMs are not possible without, effectively, the entire recorded intellectual output of human civilization since the dawn of recorded history as training data.
Trying to determine who “owns” that data, or each individual piece of it, leads us to insane distortions like LLM labs scanning and destroying books. Nobody, including the labs, thinks this is the best solution, but they’re doing it to comply with today’s interpretation of the law.
The reality is that if we are considering the entire corpus of training data, we all own it. We’re not really talking about who owns one book. We’re talking about who can be said to own all the knowledge in all the books. Who can be said to own the existence of books, or the conditions of civilization that cause us to value and preserve books in general.
The preservation and archiving of knowledge that enabled the emergence of LLMs is the product of a system of social relations that belong to the entire civilization. It is a commons. And when we talk about the future impact of these systems, we’re generally talking about taking the value of that commons, and concentrating the value of it in the hands of about a dozen people.
To me, there is no other way to think about a situation like that, where a tiny handful of people realize all the surplus value latent in the entire intellectual inheritance of human civilization, as anything other than theft.
One-off settlements with individual rights holders don't fix that. Neither does scrambling the output of LLMs so that they don’t reproduce their training data verbatim. I don’t think making LLM output non-copyrightable, or all LLM output open source, fixes it either.
I think we need to establish an entirely new framework for thinking about this stuff, and then go forward from there.
> The preservation and archiving of knowledge that enabled the emergence of LLMs is the product of a system of social relations that belong to the entire civilization. It is a commons.
This is the right framing. LLMs are a marvel built on the commons. Unfortunately, these wondrous information synthesis engines are currently captured and controlled by a tiny insane elite.
They deserve to be rewarded for their hard work, the risks they’ve taken, and the capital that was invested. However, the output should belong to civilisation. What should be open source, is the models themselves.
The problem with this is the assumption that models used are not a factor of the code already written by the user and ingested as context to determine the future state of the code according to the specifications.
All purely AI-generated code should be Open Source.
I've got plenty of technical designs that should not be Open Source and follow distinct design decisions AI can work with, not necessary know how to deal without context. If AI is additive it is just adding on the existing design not changing it.
Also I would dislike AI rot following into the codebase of Open Source projects making them unmaintainable.
End of the day question is if AI is driving you or are you driving the AI?
Doe v. GitHub already decided (I'm simplifying, read the actual thing yourself) that LLM output is not automatically open source - or specifically it doesn't infringe on the open source training material and doesn't inherit the oss copyright notice.
Thaler v. Perlmutter (again simplified) showed 100% AI output can't be given any copyright.
The directions the current cases are going are very much opposed to that idea and I wouldn't expect things to change in the opposite direction.
That is not the Thaler v. Perlmutter conclusion. That case was about whether the machine itself could be made an author under copyright law. That is very different than whether AI output can be protected by copyright law. I am not aware of any cases that test that question yet.
They will just claim the code is trade secret and use Defend Trade Secrets Act (DTSA).
And, the DTSA has only two reqs:
derive independent economic value from not being generally known or readily ascertainable by others, and the "owner" takes reasonable measures to maintain their secrecy (e.g., coke secret sauce).
since trade secret law does not prevent independent development or lawful reverse engineering, there are still opportunities. also, if a someone independently devs the same functionality, trade secret generally does not give them original dev ownership of that functionality. so, win.
Artcraft will be a good test case if Adobe goes after Brandon Thomas.
Much to be developed.
(and, this is why we keep extensive commit and development records outside of our genAI - ie provenance tracking)
Unless you only use the code in-house and never distribute I don’t see how you could argue that it’s a secret?
Remember that's in the US, most other places (UK etc..), it can be copyrighted.
I doubt that would hold the test of time, and any ruling that way will be overturned. The Berne Convention leaves what constitutes "authorship" as an exercise to the signatories, but the US probably has a lower standard.
US: "a modicum of creativity"
EU and UK: "the author's own intellectual creation"
If a court ruled in favor of AI authorship, that can't reconcile with the existing standard or with international treaties, and will likely not stand.
Unfortunately, even if other countries have different rules, the US service like GitHub still has to follow the US law and DMCA.
People who upload or download code from a US service also have to follow the law in their own countries.
Github also has to follow the law in any country where it allows usage of its services. Not all countries can enforce that but anywhere it operates or has enough paying customers can
AI generated art with human steering is copyrightable [1,2].
Agentic code development is human steering, thus copyrightable under the same grounds per the language used by the US Copyright Office.
Open source code is copyrighted, otherwise we would be talking about public domain.
When you edit something in the public domain to enrich it, that derived work then becomes copyrightable again.
[1] https://journals.law.harvard.edu/jsel/2025/03/u-s-copyright-...
[2] https://news.artnet.com/art-world/invoke-snags-first-ai-imag...
---
Edit: open source software itself has only been lightly tested in court. But that isn't even my point here.
The US Copyright Office has assigned copyright to works that leveraged AI. The USCO stated,
> “[we] decided to register a copyright claim in [the] work because we find that it contains a sufficient amount of human original authorship in the selection, arrangement, and coordination of the AI-generated material that may be regarded as copyrightable.”
Copyright office activity is orthogonal to litigation and the court system.
If a work receives a copyright, the courts will treat it differently if it becomes the subject of litigation.
Even open source license claims have not been fully litigated. We don't have to wait on the court to make certain assumptions.
You can't apply naïve transitive reasoning to law. Law is not code. Your assertions are meaningless and unsupported noise until they're actually tried in a court.
Was your assertion tried in court? legal reasoning is naïve transitive reasoning.
a statement *about* the legal system can be true without being litigated *in* the legal system. Courts rely on precedent in addition to logical argument, so their rulings cannot be deduced from rational argument alone.
I'm reporting on what the US Copyright Office is saying. Not the court system.
The US copyright office is not the body that owns interpretation of the law though. They don’t determine in court if an artifact can be protected by copyright law.
The USCO is the regulatory and administrative authority and will inform the decisions of judges. They provide expert legal opinion that carries substantial weight.
It would be a surprising result for the courts to overturn the USCO. The courts routinely side with the USCO.
What the USCO is saying is a temperature check on how this will play out.
fwiw, I suspect that a "low effort" AI-generated artifact will not be protected by copyright law. I suspect there will be a line somewhere, and it will be very unclear where it is. That said, I think it is important to be clear that this is uncharted territory, and while everything you said is true, the USCO does not establish precedent.
> It would be a surprising result
Yes, but also not the craziest thing the courts have ruled.
People show completely opposite reactions to AI generated creative contents vs technical materials such as source codes and instruction manuals.
If you think about the distinction between those, it is often so considered that both are covered by copyrights, despite technical definitions of copyrights never covering mere technical resources, such as some computer programming things and instruction documents.
Only contents of artistic values are supposed to be covered by copyright laws.
IMO, what AI revealed is that most copyrighted software and documents are in fact not covered by copyright laws as per the naive model of it among common folks as well as legal text, as opposed to what precedents say. Code is not copyrightable.
I don't necessarily support that stance from practical standpoint, and I'd rather like everything be licensed under AGPLv3++ than entire concept of code copyright fall apart overnight, but this feels like the writings on the wall to me.
Your first cited article doesn’t necessarily support the idea that agentic code generation is copyrightable.
“A Single Piece of American Cheese sits between those contrasting poles: the initial image was created entirely by the AI program based on a brief text prompt, then the user repeatedly selected elements of the image and used additional prompts to add/alter elements.”
Agentic development is steadily reducing the necessity for human involvement in the final output.
It's legally untested, so we can't be sure one way or the other. But people definitely seem to assume that's the case.
Every major corporation on earth is behaving as if it was. If/when it gets tested you can be sure that the pro-copyright side will be very, very well funded.
I think most expect the courts will land on training AI on copyrighted works is fine, as long as the output of the LLMs is transformative enough to not violate the original copyright. Exactly the same as if a human learned on copyrighted material and outputted their own artifacts.
Why should we grant any human rights to a machine that's owned by a corporation?
It depends on the degree of human steering. Vibe slopped code is probably not copyrightable.
This is very much still working it's way through the courts (in the US at least).
The first major precedent was in 2023 with Zarya of the Dawn comic book, where the US Copyright Office rules the midjourney-generated images were not protected, but the human-written text was.
The active front is Allen v. Perlmutter.
There's still something dark at the center; a lawless viciousness to the new era.
I really have to wonder what context this person is operating in. The vast majority of software written in the world is proprietary, written for a company to aid some business process, and ridiculously specific to its use case, and freely intermixed with proprietary business information.
“Open sourcing” is not just a legal state, it’s an affirmative act that takes work to prepare the codebase for public consumption. Saying that all the weird business code in the world should be involuntarily publicized is an incoherent concept. What precisely would OP expect to happen in a world that adhered to their opinion? Because to be clear, 90%+ of all code being written today is LLM-generated, so OP suggests that companies should no longer have any privacy wrt their internal software, all LLM code just gets yeeted into the public. How could that possibly work?
Given how easy it is to write software now, this perspective seems to overindex on a past state of the world that no longer applies.
> Given how easy it is to write software now, this perspective seems to overindex on a past state of the world that no longer applies.
This opinion by OP also assumes that code itself has some form of value. With the ways LLM work now code by itself is nigh-on worthless and as the models are becoming more capable, the value of code keeps going down.
Companies wilfully betted on AI, usually aware that there were risks.
Maybe companies should just stop using LLMs for 90% of the code?
The article contradicts itself.
It's important to remember that the purpose of the license is to allow the copyright owner to exercise their rights in a manner they deem fit. Even if they choose to give the code away, that choice is captured in the form of a specifically worded license (MIT, BSD, etc.).
This means that licenses exist because code is copyrightable in the first place, with the license governing the use of said copyright. If AI code is not copyrightable, then no license can govern it.
> If AI code is not copyrightable, then no license can govern it.
This is the crux of the issue. A license controls usage of copyrighted intellectual property. Until the underlying copyright/ip issues are resolved, it seems a moot point to discuss licensing.
That said, if AI code is _not_ copyrightable, does it automatically go into the public domain? Or is there a third category?
> That said, if AI code is _not_ copyrightable, does it automatically go into the public domain? Or is there a third category?
It means that AI-generated code, including proprietary code, is not covered by the existing license mechanisms, including EULAs. Companies will of course handwave over this, but it would not surprise me if this is challenged in court at some point.
Maybe the Do What The F*ck You Want To Public License (WTFPL). Its a super-permissive open-source license that acts almost like a public domain dedication.
The distinction is between creation and distribution.
Even if you accept that AI code isn't protected by copyright, code isn't public domain unless it's released. The legal issue would likely revolve around what 'released' means.
Code that runs on private servers isn't technically released. Code in web pages technically is.
But ultimately legal decisions decide to a large extent on who has the most money.
The frontier labs may eventually try to argue that they own the IP of all code, content, and inventions generated using their technology. That argument may or may not stand, and it's more likely to be over patent rights than plain copyright.
But there's a fair chance they'll try it once AI is unavoidably embedded in everyone's workflow.
Copyright applies (or doesn't) from the moment of creation, as per international treaty in all Berne Convention signatory countries. Distribution is irrelevant. Otherwise the law wouldn't serve the original intent of preventing authors from being ripped off by publishers.
"Public domain" is a synonym for "copyright does not apply or no longer applies." There's nothing special about it beyond that.
Code without a copyright can generally be distributed and put in the pubic domain, and I wouldn't bet that the few existing court decisions won't be reversed.
But, the license reflects the rights that a copyright owner granted you; I don't see why laundering copyrighted material with training a model should remove the limitations on the material's usage that the author established.
So, while it makes sense to not be able to attach whatever copyright you want to LLM output, if it's the result of training on copyrighted code, the extent to which you're allowed to use it should be the intersection of the rights granted to you by that code's copyright owners.
It should thus carry a license that reflects the intersection of those rights.
What makes it much worse is that the model trainers don't tell you what specific sources determined the output, so you should probably just assume that it's the result of its whole training, and the only rights you have are those granted you by all the authors of the training material; and, this in all likelihood means no rights, for most LLM models in use now.
LLMs are mostly trained on works that are not open source, by your own argument any LLM generated work couldn’t be open sourced.
Couldn't be used at all, rather
> It's important to remember that the purpose of the license is to allow the copyright owner to exercise their rights in a manner they deem fit.
Just as it's important to remember that the purpose of regulations like copyright and patents are to promote the progress of science and the useful arts.
We're not locked in to a suicide pact with what the Founding Fathers decided back in the late 1700s. Congress can decide that all LLM-generated code is automatically covered by -say- the GPLv3, even if no one holds the copyright to it.
If the absence of a particular person to hold the copyright twists your knickers, you could go a bit further and -say- decide that the people of the United States collectively hold the copyright to all LLM-generated code that's generated by US people or entities, and that relicensing of a particular piece of LLM-generated code must be agreed to by every US citizen that was alive when that code was created.
> We're not locked in to a suicide pact with what the Founding Fathers decided back in the late 1700s. Congress can decide that all LLM-generated code is automatically covered by -say- the GPLv3, even if no one holds the copyright to it.
I have no issue with changing the law. My comment is about the article and the author's misapplication of the law.
> My comment is about the article and the author's misapplication of the law.
What parts of the six-paragraph Fine Article misapply the law? Be sure to quote and include the context surrounding the quoted section(s).
I think it would be the part about how courts have a fairly strong status quo bias and aren't going to remake the entire economy overnight with completely unenforceable provisions.
What are they going to do confiscate the world's gamer GPUs that can run this stuff all locally?
It's completely ridiculous.
This is the way
> you could go a bit further and -say- decide that the people of the United States collectively hold the copyright
Was going to comment something similar; we do have concepts of "dominium eminens", and "escheated property", and "public domain", and many other similar ideas. If the people (through their representatives) decide on a certain property regime, it absolutely can be implemented even if it's different from the status quo.
> even if no one company or person owns it.
Maybe, but I don't think there's any legal president for that, is there? Much of existing law is based on centuries of private property rights. I don't see any government undoing that... or at least, not quickly.
> Maybe, but I don't think there's any legal president for that, is there?
Article I, Section 8 of the Constitution and its various subsections pretty clearly gives Congress this power.The IP laws are a) not really modelled after material property rights; b) are quite much more recent than "centuries"; c) there is legal precedent for that, see e.g. the airplane-related patents being pooled in 1917 by the demand from the US government.
Nobody is going to interpret the laws that way. That's completely stupid.
Everyone is using AI generated code in all sorts of ways, and the idea that the courts are just going to say "nuh-uh. All existing use of AI code is illegal" is completely ridiculous.
Courts don't upend entire economies like that. This would be the remaking of the entire software industry overnight and it would be completely unenforceable.
I'm certainly not going to listen to any such ruling and will continue making code locally. As will basically every other software engineer in the industry.
> Nobody is going to interpret the laws that way. That's completely stupid.
You're absolutely right! And a little more than two hours before you posted this comment, I posted this [0] one! Make sure to read the parent comment for the context... it's pretty important.
[0] <https://news.ycombinator.com/item?id=50507914>
Gotcha, not really related to my point that the courts aren't going to completely remake the entire software industry over night and neither is congress.
That's absurd. It is a fantasy. Neither lawmakers or anyone else is going to remake the software economy.
> Gotcha, not really related to my point that...
Your point was entirely off topic, and the retcon of your intention is weaksauce.
> It is a fantasy.
Reread the title of TFA.
> Neither lawmakers or anyone else is going to remake the software economy.
LLM-generated code is -when measured in regulatory timescales- brand new and is -according to boosters- "remak[ing] the software economy". The time to make new law and regulation to make it prosocial, rather than profoundly antisocial is _now_.
Kind of a silly position. I mean, if it's you're code verbatim, then yeah, it's a violation. Shouldn't be allowed. But like... it saw open-source code and therefore all output must be open source? I don't know about that.
Here's where I can get on board. I think people should be putting AI to work to improve everyone's lives for free. Your version of "should" is like "we should make this a law." My version of "should" is more like a moral recommendation -- we should be using as much of our subscriptions as possible to benefit everyone instead of enriching a handful of people. I view it as a natural extension of the spirit of free software to begin with. Software should be free (as in freedom), and we finally have enough development power to make real headway on that.
I have some projects I've been working on over a year. Sitting down working making design decisions. Research. Using ai doesn't mean I did no work. It doesn't matter if the ai was trained. I still did work.
I don't understand this view point. Using ai and making high quality stuff is not pushing a button. Its hard. It takes real effort and diligence. And you have to pay actual money out of pocket to deliver it.
What if I am creating something that does not exists yet using AI? So i did the research, layout the architecture, decide on the implementation details, write the documentation and the rules AI has to use to create the software, because I am only one person and I can't physically write all that code on my own on a timely basis (it would take me years to write all that code, test it and debug it until i am satisfied)? I am creating something new, not copying anything, so where I stand? Can i publish my code, say with MIT licensing, and retain the copyright?
> I have some projects I've been working on over a year. Sitting down working making design decisions. Research. Using ai doesn't mean I did no work. It doesn't matter if the ai was trained. I still did work.
> I don't understand this view point. Using ai and making high quality stuff is not pushing a button. Its hard. It takes real effort and diligence. And you have to pay actual money out of pocket to deliver it.
You seem to believe that because you put months of work into this, you should have rights over the result, right?
But once you release, anyone can spend a few dozens of dollars and replicate it. Do they get the same rights over the clone that you got over your product, even though you potentially spent months coming up with it and they cloned it in a day?
There's really only two answers to this - either you agree that they should get the same rights over the clone that you got over your product, or you don't agree that they should have the same rights over their clone that you have over your product.
In either case, you are not better off:
1. If they get the same rights over the clone that you get on your product, you can't really stop people cloning your stuff, adding a feature or two, removing another feature or two, and then release something better than you for 1/10th of the cost of production.
2. If they don't get the same rights as you did because they used your product as training material for a clone, then wtf do you get any rights at all, after using millions of other people's products as training material?
I don't see any scenario where there is an outcome beneficial to you. Do you?
This is a false dichotomy. In this example the person you’re replying to isn’t wholesale copying someone’s work.
> This is a false dichotomy. In this example the person you’re replying to isn’t wholesale copying someone’s work.
So? I said "clone", but clarified that features may be added to the clone to make a better product.
Taking his product $FOO, adding it to my training set/context/RAG/LORA and/or tool call to explore his product in detail before I tell my LLM to produce $FOO+1 is exactly the same thing that he did.
3. Others are free to use the produced material as educational/training material if they can gain access to the code to then use it for their inspiration. But they are not allowed to outright duplicate it.
This seems the obvious answer? No? The exact clone at 1/10 the cost is just really unlikely imo, especially if it's closed source. And with open source this was already a problem anyway, you could always fork, but hey, turns out most people just stick to the original anyway
> No? The exact clone at 1/10 the cost is just really unlikely imo, especially if it's closed source.
The availability of the source is irrelevant; I posed this question with the understanding that I could simply add your product, whether closed or not, to my LLM, and get it to produce a better product without all that pesky effort and token usage you used.
Release your product. See how quickly it gets "cloned". i.e. copied and reproduced without anyone needing the source code.
All that effort of yours... just to result in someone else producing the same product in a different language, but with all your product decisions already baked in for nowhere near what it cost you.
> And with open source this was already a problem anyway, you could always fork
Sure, and people famously did so, but they could not clone original closed-source products without a lot of effort, often taking months to years. Now, the result of you spending years on perfecting a product means that a clone can arise in a matter of days.
It also takes effort to copy something, right? Say I made a copy of a book by hand, does my effort make it any less of a copy?
Say I am stealing stuff that’s really heavy, can I demand I earned it because it was a lot of effort?
I am not trying to be facetious. I also question property rights because someone overpowered someone else to make that property theirs at some point in time. Whose effort is earned, and whose is robbery?
As far as I can tell we picked an arbitrary time after which all forced acquisition is considered robbery.
That being said, open source is for everyone to benefit. Violating an open source license is sort of a crime against everyone, not just the creator.
Absolutely. There is so much variability in the code that can be generated for a particular purpose. I do hundreds of iterations before my LLM-generated code reaches the desired shape. The built harness, the written AGENTS.md and the iterations affects what is written greatly.
And at least of the level of feeling the answer is that it is my code.
It's unfortunate that you had to pay actual money and effort, but there was also actual money and effort involved in the projects on which the LLM output is based.
When a carpenter crafts a chair, the tree first had to make effort to generate the wood.
Because chairs are in effect an open source design, does that mean carpenters shouldn't get paid?
Its nonsensical.
I think that people who believe these arguments have bought into ai hype too much and genuinely are fooled into thinking pressing a button to spit out boiler plate code somehow equates to a genuine saleable product. It doesn't and it won't. And that stuff shouldn't be open source because it's a waste of everyone's time to consume.
The difference between the input to the carpenter and the output of the carpenter is significantly greater than the input from you to the LLM, and the output of LLM. Majority of the alterations to the wood by the carpenter produced the chair. The words you typed into an LLM are less than 0.000000001% of the total inputs that it took to create its output (because training data counts, and it’s a lot bigger than anything you may have typed)
Yes, what you wrote is definitely nonsensical.
Chairs can be covered by design protection.
And however hard a carpenter might work on a chair, if he copies a protected design he can't sell the chair.
However much work you might add on top of other people's copyrighted code, you don't get to sell it and use it however you want if you don't have the permission of the original authors.
> However much work you might add on top of other people's copyrighted code, you don't get to sell it and use it however you want if you don't have the permission of the original authors.
It gets even more hilarious when you realise that the GP (tim) does not consider that, if you make the output of AI that was trained on copyrighted material legal to re-copyright, then it applies to him as well - all that hard work he did can be used chucked into an LLM either as context, or via LoRA, or even via simple explorative usage via tool calls, and then regenerated from that LLM, taking away whatever copyright the product had by generating a competing clone.
https://news.ycombinator.com/item?id=50509402
Strongly disagree.
Just because AI was involved doesn’t mean significant human thought and effort didn’t go into the production of those artifacts.
My agentic software development loop requires heavy involvement of real-world feedback. The ratio is huge. 20 mins of agentic software coding against hours or days of testing, trying it out, real-world contact, thinking about what feature is next or what tweak would make it more usable, smooth, intuitive. That feedback is baked into the product. Forcing it to be open source steals all that time and effort for me, and takes from me the decision as to how and whether to open-source it. So you mean the code that I’ve agentically written to help manage my private financial affairs (on local models, for privacy) or my kids’ complicated school schedules must now be open sourced?
My manners begin to fail, so I’ll stop here.
US legal practice (e.g. https://en.wikipedia.org/wiki/Feist_Publications,_Inc._v._Ru....) is that "sweat of the brow" is irrelevant and the fact that something took significant effort doesn't mean that the product deserves copyright protection.
Furthermore, copyright protects only the expression, not the idea - reimplementing the same idea with "different words" has always been permitted under copyright law (patents may protect the idea, though).
Of course, the fact that it would lack copyright protection it doesn't implythat you have to do something with the code for management of your kid's schedules, just that if someone else somehow legally gets hold of it, the law won't prevent them from making and distributing copies or derived work.
Thought production isn't copyrightable. Processes and procedures aren't copyrightable. Might get you a patent.
> Forcing it to be open source steals all that time and effort for me
You are the one "stealing" the time and effort of thousands of people who have written free and open source software. If the ratio really is so huge, then just cut out the small part that does all of the stealing.
> So you mean the code that I’ve agentically written to help manage my private financial affairs (on local models, for privacy) or my kids’ complicated school schedules must now be open sourced?
Open source licensing is only relevant when distributing the software, if you just use it privately it has no effect. You are not forced to make it public.
That would like require learning to code. No one wants to do that anymore.
> My agentic software development loop requires heavy involvement of real-world feedback. The ratio is huge. 20 mins of agentic software coding against hours or days of testing, trying it out, real-world contact, thinking about what feature is next or what tweak would make it more usable, smooth, intuitive. That feedback is baked into the product. Forcing it to be open source steals all that time and effort for me, and takes from me the decision as to how and whether to open-source it.
It doesn't need to be open-sourced: if it's legal to take an LLM that was trained on every other product in existence and use that LLM to produce clones of all those products, then it's legal to do the same to your product to.
So, go ahead and release. There'll be an open-source clone in no time if the product is any good at all.
In fact, if there are no clones, take that as direct feedback that you wasted your money on tokens - your product is so bad no one even wants it for the few dollars it will take to clone it.
> Just because AI was involved doesn’t mean significant human thought and effort didn’t go into the production of those artifacts.
Doesn't mean it did either. (99% of the time it didn't tho. Nobody reaches to a vibe code machine because they care or want to dedicate effort.)
I don’t know about one-shot-prompt toy apps, but when it comes to building real stuff with AI, it still takes significant effort from devs to get the results we need. There’s a difference, right? “Give me the best result” vs “Give me the result I want”. Most serious projects, at least from what I know, run on the “Give me the result I want” principle, and that’s where the dev effort goes. I know the future is heading toward “Give me the best result with no mistakes”, but that changes how much effort it takes, not who owns the output. Even if the code is generated by AI, humans own the result, because they worked with the machine to get the result they wanted. Same was true before AI: humans worked with the machine to get the result they wanted. It’s just way faster now. Using a machine to build something has never meant you don’t own what you built.
Nearly every anti-AI argument seems to be someone who’s limited experience is one-shotting.
Most of my AI experience takes quite a lot of supervision just to get the model in the context to be able to make changes effectively.
The other take is AI code generation machines are directed by human inputs and will never start without that human input, so therefore it was created by a human.
Much like how a photographer 'just' clicks a button on a machine and through a very complicated process it creates media, so does the human who makes media via an AI machine.
I think because of how corporate power works and how much money is at stake this will end up as the end result. Laws are made up by humans and a majority vote in a parliament can set this to be so, previous rulings be damned. And most treaties never really ever think about how media was made because this was never an question until recently. Copyright are not constitutional rights in most counties.
Matthew Butterick (of https://typographyforlawyers.com/ fame) is involved in a class action suit about LLM being trained on copyrighted material. See https://llmlitigation.com/.
Maybe we should get some high-profile software devs on that as well ...
Actually: All AI-generated _Output_ should be Open Source (not just code)
It's the (open) knowledge of humanity, and the thin layer above is only a minor fraction (0.1 if at all) of the 'product', which should not be allowed to be privatized by anyone - let alone destroying originals (like books) to own 'it' !
This article is mostly ragebait, and it's claim is hollow when looked thru the lens of computing history.
Let us skip the whole Bell Labs C/C++/unix/sh/ksh -> BSD csh/vi/bsd/tcp/ip, and other proprietary unix variants/utils and focus instead on today's landscape.
There are two axes we can partition the whole ecosystem. Free/OpenBSD, Clang/LLVM, musl, busybox, X11/Xlib, sqlite, postgresql, nginx, Python, Go, Rust, many embedded rtos, vendor sdk, utilities etc.. and then there is Linux/GCC/glibc, etc. Along the licensing spectrum there is GPL and then the rest of the permissive licenses MIT/BSD/Apache etc where source code publishing is not a requirement.
These permissive license offerings are more prevalent compared to the GPL offerings and as such one can gain all the knowledge needed to build a fully functional system. LLM could memorize, or copy the pattern, so even though it is not a clean room, courts seem to be ok as long as things are not spit out verbatim.
Add to this, the fact the there are tons of research articles, user manuals, RFCs, so one can learn and understand how to build the needed reusable component if small and give their own prompt (and bugs) and and let LLM do as user described.
In the US, purely AI-generated code isn’t copyrightable, but it also doesn’t mean you have to publish it. Also, putting an open-source license on code nobody holds a copyright to has no legal effect. So, ethically or otherwise, there is no strong and convincing argument that open sourcing is the right thing to do.
What 'creators' could do is to compile a list of all the frequently used permissive licenses that are applicable to their system and just make it part of their software license attribution.
[1] https://redmonk.com/sogrady/2026/03/25/open-source-licensing...
[2] https://arxiv.org/pdf/2211.15533
Whilst the first training sets would have used Stack overflow and open source projects.
I have my suspicions that the AI companies have been using reverse engineering to gain more training data.
Ethics don't really concern these companies so...
When you have essentially infinite compute and are constantly looking for more training data. To me the next logical step is to start reversing compiled programs.
It could also explain why these AI models are so good at it too...
How do you copyright stolen material?
Reverse engineering is legal so I'm not sure what you're trying to imply here
i think the author presents weak arguments:
> AI models were trained on a massive corpus of Open Source
using open source shouldn't require the final product to be open source. that's pretty much most of proprietary software. most of it is built using open source code but the final product is due to a combination of thoughtful product and software knowledge and must be private since it's really the value you produce.
> purely AI-generated code is not written by a human, meaning it should not be copyrightable.
you can say AI generated code is based on human code, and that human code is open source. depending on the license (likely MIT) its technically legal to choose the generated code to be private.
> To make things fair across all jurisdictions, and to honor the open ecosystem that made AI possible, developers and companies should explicitly release AI-generated code under Open Source licenses or dedicate it to the public domain using the Creative Commons Zero (CC0) license.
i somewhat agree here. AI generated code should include the original license of its authors unless it's purely AI generated using original variable names.
i previously released AI generated products/software as open source but come to think of it, in the world of abundant software, i believe there's more reason to actually privatize your code vs putting it out in the public. especially for paid software products.
Not the AI's source code output should be open source but the prompt should be. The prompt is the real source and can be used in the future with newer LLM models.
> purely AI-generated code is not written by a human, meaning it should not be copyrightable
There's a lot of room in between code that is closed-source and code that is "copyrightable".
Should code crafted via cut-and-paste from StackOverflow be open source? How about newsgroups before that? What about apps that use "no code" tooling? Is there a threshold for the use of external libraries and packages that cross the threshold for "that code was simply scraped from the internet and used without permission or proper attribution"? How about code I wrote while using auto-complete a lot?
I appreciate the author's sentiment and recognize that LLMs have aggregated the work of many, many people. I'm guessing their work wasn't wholly original either, like so much of the code I wrote before the age of LLMs. Whether those people whose work has been aggregated to train LLMs own that work - those ideas, those thoughts, those strategies - is exceedingly complex and (I speculate) largely won't hold up to real scrutiny.
I mostly agree and have pretty much every one of my projects open source. One in particular, though, is not. And that is because the design of it is heavily domain specific and is built on loads and loads of experience. Not sure how to reconcile that...
There's a difference between something being released as open source and something that never gets released at all. Open source or not only becomes relevant when making the binary available to other people.
Yes, but the one project I'm excluding from open source may be one I sell to companies - we'll see.
Good link in the post https://commons.wikimedia.org/wiki/Commons:AI-generated_medi... happy but not surprised that Wikimedia Commons has a good approach and I lovely template name as part of it: PD-algorithm
If you type in something like: the history of code and copying previous code versus learning from it and applying that to creative intent behind AI usage for code creation --- you will find that historically that has been a progression from day one. The effort to get good code out of an AI and test/validate it does not minimize the creative intent of the user. Now intent is the operative work. Some intent is bad, some is good. Determining intent is hard for humans as it is for AI. I don't think there will be good answers except for individuals. The courts may rule one way or another. Public opinion in different groups will sway one way or the other. Saying anything is this or that is a slippery slope. The whole premise of this small article seems flawed as a concept and just ignorant of the history. Much like suing someone for a musical phrase of a few notes. There are the same chords/melodies in countless songs and it is not done maliciously. There are only 12 notes, so you figure it out. I think everyone is going crazy about AI on both sides. There are a large group in the middle that are learning how to use it and what it is and is not good for.
There's a decent change that AI makes closed-source software not really a thing anyway. So it'll either be open or unreleased.
I’m confused. Is this article meant to be a scholarly legal argument (albeit, by a novice)? Or a normative statement about what the author would prefer?
It certainly doesn’t seem to be about practical solutions. It doesn’t even touch on the ramifications of its suggestions. I’m pretty sure that it would be a massive boon to data center operators and cause most of the readers of hacker news to lose our livelihoods.
Personally I am less concerned with all AI generated code being open source. We will very quickly be drowning in a sea of code, and who even wants to see it if it's easily re-created by AI? Eventually everyone will be using AI and I don't think it's feasible or makes sense for all code to be open source. Personally I _would_ like to see all models trained on public data become open-weights. To me it seems unfair to let a few companies lock up all the value of the collective works of civilization without providing something back for free.
Two points I think making AI-generated code opensource Point 1: AI can (or will be able to) reverse engineer all code from how the program works. Thus, there seems to be little point in opensourcing AI generated code because it can just be reversed engineered
Point 2: Correct me if wrong but can AI generated code actually be copyrighted? If someone just keeps prompting but not touching the code would this still be considered pure AI generated work? If this is the case then point 1 and point 2 basically means AI generated code is already open source in effect if not in name.
A lot of y'all here are missing something important: open source code is an application of copyright law, and it has already become clear in some countries that the output of an LLM is not coverable under copyright (i.e. it is public domain).
In the UK, the output of an LLM can come under copyright and therefore the author can license it as an open source product, but in the US it can't be done.
Related from the FSFE, with a bit more of legal background: https://news.ycombinator.com/item?id=50489088
No, but all models should be open weight.
Since last year everything I was generating with AI I put it opensource on github! Some of these tools I use on daily basis others were experiments. It felt natural to do so, I never for a second felt or thought I owned the code output of LLMs even if I am the one who prompted it
I can understand the case being made on "purely AI generated code."
But thats also why its buggy.
As soon as I start refining it, and commanding the AI to fix the mistakes, this argument starts to lose ground fast.
The tech community has adopted open standards for as long as I've been alive, and the AI is using these open standards as much as it is actual code.
Perhaps an open standard to prevent code from being used as training ( it probably already exists, i dunno ).
Methods to detect it ( I've seen that showing up recently. Using AI to detect AI. )
Some sanctioned actionable penalty under provable circumstance as the resolution ?
You can see why the copyright office wants to treat this as any other thing.
The details of proving claims get complicated quickly.
> As soon as I start refining it, and commanding the AI to fix the mistakes, this argument starts to lose ground fast.
Definitely. And if the result isn’t essentially a clone of some existing copyrighted work, there really is no viable argument under current copyright law.
Couldn't an org still constrain distribution (keep the repo private forever)?
At this point, since LLMs can reverse executables anyway, making something Open Source is hardly any leverage. Instead we could say that any products generated by AI face a tax.
Actually, why not go straight to the source and tax the AI companies for the public benefit. That is the logical reduction of the argument.
He’s not wrong, but I would argue a better model is the Derivative Work. This already exists in copyright law.
If I write a space opera that is a LOT like Star Wars there are major legal implications to this. If I write an original work of science fiction involving a weapon that destroys planets as an allegory to the history of the atomic bomb, I’m probably in the clear even if one of the characters is a furry alien with a laser sword.
The challenge now is how to apply that framework when nobody can see the original or resulting code.
Derivative works still have their own copyrights, though. For instance, Kouta Hirano might own Hellsing, but that doesn't automatically mean that he owns enthusiastic walks through the woods, "bitch I eat people," or High School of the Dead.
By that logic, LLMs that were trained purely on proprietary code should be allowed to generate proprietary code (Microsoft, Oracle, Apple, etc. should have a quite large codebase when combining all their products).
And then again, I wonder which license should be used. Is this a case of copyleft, and therefore everything generated should be something like GPL, or is it more like public domain because no copyright can be exercised?
Alternative view on the problem. if code is really a solved problem , open source becomes a security problem? with all the attacks happening mostly on open source projects, it is not farfetched that a secure bank system will be ai generated with core libraries implementing everything from scratch, including runtime. Big business, big problems.
This idea of no libraries/external dependencies sounds extreme, until you got hacked by a tiny library for http calls and that cost you billions of losses.
Do we still care if its open or closed? Isn't reverse engineering basically solved anyway?
What's wrong with this website? When I open it in Safari on macOS, it consumes all my RAM. 20GB to start. I got a system warning to force quit some apps to maintain stability (which was a first in 5 years). When I closed other non-Safari apps, Safari usage then went up to 30GB.
Quitting and relaunching Safari, then returning to the site, produces the same results. The process for this website is what is using the RAM, it persists after closing the tab, and needs to be force quit.
I disagree, however companies who decide to work on proprietary code need to be doing comprehensive license similarity scans on their codebase or risk licensing issues
Definitely agree with that in concept, but there really a fact that nobody realized until recently after all the game decompilation/mix-ups: LLMs can read assembly fairly well and traditional obfuscation techniques doesn't really work on them, so if you have access to the binary and a disassembler, an LLM agent will be able to reverse engineer and reconstruct the likely high level source code of most software given enough time anyways.
In other words, almost nothing is closed source to LLMs if they have access to the binary.
The article is right. If no human effort is involved then the code should be open source. The article also implies a very detailed line too. It doesn’t say that the human effort part can’t be copyrighted or intellectual property.
As an example, I have developed a set of grounding guidance for my AI models at my work. The specific grammar and vocabulary belong to my employer. But I also have a clean-room version of it that completely strips out that proprietary infringing information while still retaining a similar semantic structure. That will be posted next month on my GitHub under a GPL-3 license like I do with my other non-infringing AI-assisted work.
In a way all LLM code is already open source. We all have open access to the system which generates the code. All you need is the prompt, so that is the non open element.
> developers more and more rely on AI without even reviewing the generated output, let alone modifying it to fit their specific needs. Code like this, where humans put little effort, except writing prompts, should definitely be Open Source.
A person's contribution here is difficult to measure. This statement reminds me of one of my older bosses who priced software per lines written.
An AI-generated software's quality reflects the author's vision, taste, and expertise (if any). These are there regardless of the number of lines reviewed.
I would've supported open source for completely AI-generated apps including the conception of the idea.
RMS would be proud with this post though.
I'm all for the concept of all code being open source... but I would then need a way to protect my final product from just being cloned or ripped off by a competitor.
With code itself being so cheap these days, it's the final product that has value. The requirements, architecture, usage data, the hundreds of meetings to figure out what customers want/need, the marketing, etc. A lot of the learnings and most of my business are expressed in the code though, so making it open is a massive risk. The code on it's own I don't even think is valuable. The only ones who'd benefit are competitors and upstarts targeting my customers.
You can protect it with a copyright. But you can't copyright machine generated code.
I agree, but also, any code that a company can generate and use, an open source solution can generate and release, effectively eating the corporation’s lunch in many cases.
> The second reason is legal: purely AI-generated code is not written by a human, meaning it should not be copyrightable
I think that's the actual point here. AI-generated code shouldn't necessarily be open source, but it is not (or should not be) copyrighted. You can still keep it as a trade secret, but you have no legal protections that prevent others from reusing it once they have access to it.
> You can still keep it as a trade secret, but you have no legal protections that prevent others from reusing it once they have access to it.
The funny thing is though that soon this isn't even really going to be that much of a need to re-use someone else's code?
Both claims are easily rebutted .
> AI is trained on open source
It sure is, but the composition is a unique creation. Every creative artifact includes some derivation. A rhythm, instrument, media etc.
> AI isn’t made by humans.
Well neither is a canvas, paint, a chair. Everything we use is made by an automated tool, many products are nearly 100% automation.
"To make things fair across all jurisdictions"
PLEASE stop right there. Nations are sovereign, they decide for themselves if/how far copyrights are enforced. Let's keep it that way. There's enough [800lb gorilla meddling in other's affairs] as it is.
I could not disagree more with this view. Some of the works that reasoning models were trained on were already in the public domain. OP's statement is an attempt to get hold of code that they never build themselves. It takes human effort and human input - even if all it only determines is merely an initial random seed value - to generate the code in question. There is no fundamental baseline and there cannot be such a baseline that would provide for such a strict licensing of AI-generated code.
It already is. Just ask AI to give it to you.
It seems pretty clear-cut to me. AI companies plundered the commons, but the commons have licenses that come with them that clearly state that their use and derivatives thereof must also include the license and be open source.
This is a chance for great good for humanity and for the AI companies to do the right thing for themselves and for all of us.
Not releasing the code as open source would be just... well... evil.
And a further question arises for me: If we don't start sharing the fruits of the machines now, then when?
Valid point but this will happen due to market forces rather than moral reasons. Anyone can write code now using AI, so charging for the code itself is getting harder every day, and only the trustworthy people will get the distribution.
The rest of the code will be open source, see how many open Wispr flow alternatives are there already, everyone has same features.
> purely AI-generated code is not written by a human, meaning it should not be copyrightable
This is a very poor definition. I use Phoenix's scaffolding all the time. So, running
means my code is no longer copyrightable since it was written by a machine?I agree with the spirit of this. I'm a longtime open source developer and generally prefer GPL for my manual code. For my AI-generated code I'm using a multiple choice license of Apache, BSD, GPL, MIT.
If you're curious, here's my newest open source which helps build Rust web services along with Axum: https://github.com/publicprojects-io/axa and you can see the LICENSE.md file.
Not if you see the AI-generated code as same as 90's java bytecode compiled by a tool (machine). In both cases, the input (AI prompts or 90's java code) is human-written. Just because a machine is involved in converting human input to some next low-level artifact, the low-level artifact does not become open source, when the human input is not.
What is user changes 1 line? 2 lines, 3 lines...? The concept is well intented but I doubt you can enforce it.
I get the sentiment, but how would you prove a negative? You can get "caught" using AI, but not having been caught does not and can not prove you didn't. Given that 99.999% of projects now use AI to some extend, the logical default assumption is going to be you are probably not in the 0.001%
Why only code? What about music, movies, images? If AI generated code isn't copyrightable, why should they be?
Open source is not always open source as we all know there are so many different flavours of it. How do you make that distinction when using AI to generate your code?
This is a debate that we need to have before it gets event more muddier.
Related question I just thought of reading this: if you are an employee at a company that you know 100% AI generates its code, and AI-generated code is not copyrightable... is that employee legally allowed to leave the company taking a full copy of the source code, and set it up as their own business?
I doubt anyone is 100% AI generating anything
Probably not. Even if the code is not copyrightable, I would assume it's still considered a trade secret.
Your employment contract most likely classifies it as a trade secret.
No, copyright protects works made by humans.
AI generated code should be (is?) public domain.
Somewhat amusingly, I just posted this... https://news.ycombinator.com/item?id=50508364
Could LLMs be considered derivative work in the sense of the GNU GPL?
I use AI to develop apps for manufacturing but I still have to think and point AI how things will work together. If I simply tell AI to develop something complex most of the time it will do a really bad job doing so. I do have a coding background but with AI I can focus on other things and use the time I’d spend coding to work on other problems.
I kind of agree, but like most software engineering takes I think this misses the point of software as a whole.
What the software _does_ is almost always more important than how it does it (save for some amazing new technique which is very rare).
There are only so many ways to write a binary search or an octree, but the overall idea behind, say, doom is unique.
Open sourcing big blocks of code generated using standard techniques isn’t that useful (since you can just generate the same ones yourself), but the concept behind the application is the important bit (which of course, is now also easy to just generate).
If anyone can just generate their own version, who cares if it’s open source?
Even code generated for government organizations with various levels of secrecy? Not gonna fly.
All 100% ai generated assets, art, music, text should be free and open source too.
Fully AI output from a single prompt is not copyrightable (in the US). Is fully AI output from a sequence of prompts not copyrightable? If it is, at what point does this change?
It’s not.
> Some technologies allow users to provide iterative “feedback” by providing additional prompts to the machine. For example, the user may instruct the AI to revise the generated text to mention a topic or emphasize a particular point. While such instructions may give a user greater influence over the output, the AI technology is what determines how to implement those additional instructions.
- Footnote 30
https://www.copyright.gov/ai/ai_policy_guidance.pdf
No offense to anybody but I do believe there is no Intellectual Property (IP) anymore in the age of AI if one wants to defend or attack it was breached.
All models should also be open source and socially owned. All of this output is the culmination of human knowledge and should be treated as such. I get this is about copyright but I don’t think people are thinking big enough with this stuff.
The code may be non-copyrightable, but the business logic may be proprietary and a trade secret.
All code should be Open Source.
A pre-cursor. All AI models should be open weights/source to begin with
The messy part in this idea is the “especially when human effort is minimal” bit.
As the cost of producing software approaching zero. The legal side go down with it.
AIs don't have copyright at all. All AI generated code is public domain
There is so little value to most code produced with models these days I just don’t see the point.
The real value ends up being found in software where there is a person behind it continuously refining it into something others will look at and say “Someone clearly paid attention to this.” Models can’t (yet) reproduce that because they can’t/don’t go interacting with users or asking bigger picture questions.
Whether someone is producing that software with a model or not doesn’t change much about the ownership picture to me: it’s the person that owns the project, and the LLM is a tool.
Once again, it all comes down to it being a tool and how it’s used, and there are no useful blanket statements that can be made about it (or how the tool’s output should be classified, legally or otherwise)
Does it matter at this point when you can generate the same code easily with the same AI model?
You can't? Do people really think software is just prompt and done? People seem to forget software isnt just writing code, its solving really problems, and architecting. Coding was a small part.
If AI services that created the code were free, I agree with the sentiment.
This post is so close to getting it ;)
Looks like the author wants to abolish copyright in a very round-about way. They are saying that AI-generated code should be released under CC0 due to complexities around the status of software copyright in various jurisdictions.
But if the concept of copyright was abolished as a whole, then licences such as GPL would also lose their power. Some people say that's a negative thing, but I do not necessarily think so, because it achieves the stated goal of the GPL, software freedom (i.e. you don't need to be granted a licence to copy and modify things)
I mostly agree with you, but public domain licensing doesn't get us software freedom by itself if the source code is not available.
OTOH as models get better at RE it may become moot.
no because nobody is required to release the source code of their derivative work absent a license like GPL that requires you to do it.
No, the four freedoms rely on copyright. If there is no copyright, any company can take the open source code, modify it as they like, and never give back the modifications to anyone. It's the equivalent of an MIT license (technically, it's more like public domain but let's ignore this detail here). That's fine for many but far from the goals of the GPL.
No copyright doesn't mean you can't keep your source code a tightly kept business secret. You can even combine it with NDAs and contracts to re-institute something akin to copyright.
> First of all, AI models were trained on a massive corpus of Open Source code written since RMS announced the GNU Project and the Free Software Movement.
I too have trained on the massive corpus of open source code. Should everything I code also be open source? Or only if I leverage an artificial neural network as a prosthesis to enhance the capabilities of my natural neural network?
> I too have trained on the massive corpus of open source code. Should everything I code also be open source?
Humans get rights. Tools don't. Are you claiming that you're not a human, but only a tool?
I'm claiming that my tools are extensions of me and parts of me. My freedom of expression should include the use of my arms, legs, printing press and language models.
> I'm claiming that my tools are extensions of me and parts of me.
Nevertheless, tools don't get rights, only humans do.
Even when the tool is an automatic tool, it still doesn't get the same rights as humans do.
Your tool does not automatically get the same rights as humans.
I say this because it is nonsensical comparing human-learning with tool-learning, and claiming that there is no difference.
You get the right to own property. Your tool does not.
You get the right to form associations with anyone. Your tool does not.
You get the right of expression. Your tool does not.
There needs to be a logical argument for why your tool should get a right reserved only for humans, animals, etc.
Unfortunately for you I created an algorithm that can generate all possible code, text, image, etc.
Makes sense, many of the programming languages are open source, other novelty things were released as open source aswell, we as programmers/software engineers does a bunch of our solutions based on public code or under Creative Commons.
What I agree with the post is specific licenses require us some specific requirements like release the code if it's a derivation of prior work!
No, because open source licenses don't cover that, and artificial neural networks are very different from your "natural neural network".
You're also unlikely to output what you trained on verbatim.
I'm tired of these child like responses from adult professionals in this industry. Do you argue just for the sake of arguing? No, your exposure to opensource code wasn't "massive" and you are clearly smart enough to know this but you had this make this comment as rage bait.
An entire generation of web developers learned by viewing the source of webpages they visited at a time when HTML and JavaScript were still nascent. I don't think setting the bar for "massive" is the way to win this argument. It's valid because the biggest difference between an LLM reading Stack Overflow and remembering content and me finding answers there and learning something is that the LLM can do it much, much faster.
The difference is, just as a child doesn't need to consume the entirety of a language before being able to communicate with said language, a human developer doesn't need to consume all of Stack Overflow, Open source projects etc. in order to start producing functional work.
I'm confused - I thought you said the response was child-like, not that a child can do the work. I'm not advocating that an LLM has child's level of experience, just that humans learns how to do things from looking at previously done work, so why is that not an allowed method for an artificial intelligence? How else is it supposed to learn?
This also ignores all of the great artists who learned by sitting at the feet of other great artists. Sometimes they extend the art of their teachers, sometimes they diverge.
A child or a human developer cant do all the thousands of different things an LLM can do either.
Do you people understand that my comment was about the parent commenter exaggeration on how they were "trained" on a "massive (OSS) corpus"? I'm sorry I hurt your sacred cow by implying LLMs need stupid amounts of data.
LLMs need stupid amounts of data because they do stupid amounts of things.
So if future models are able to generalize better from much smaller amounts of data, you think that all the current concerns disappear? It doesn’t seem like a very good argument if it’s purely based on the amount of data needed to train a model.
What matters are things like where the data comes from, how similar the output is to the input, how close the result is to existing (copyrighted) work, and so on.
Simply using copyrighted work, even all copyrighted work, to train a model doesn’t, by itself, trigger any legal consequences. In that respect at least, the comment you called “child like” was correct.
My comment was simply about the parent comments exaggerated claim. I do believe it should have some consequences, what those should be I am not qualified to decide.
I have learnt to read by training on a lot of books, I should then open source all future things I write. I've learnt to walk by watching other people, I probably should publish videos of me whenever I walk.
What problem does this solve exactly? I think when most code becomes generated it’s all from the same hivemind of the human collective. I see arguments that code is not the hard part of development, so why are we treating it so sanctimoniously now?
I completely agree with the OP. What irks me the most is that Amodei has trained his model on someone's else IP and now he wants to resell this to people and has the nerve to complain about distillation. Distillation should be god given right. I guess that is why they took down Jensen.
There is no such thing as AI generated code, just as there is no such thing as keyboard generated code. The only difference is that your keyboard manufacturer never had the gall to sign git commits -qwerty.
When AI becomes completely free to use -- that's when I'll support the idea that the result should be open source.
The answer is obvious. If you want to protect your code, invent a new programming language that does weird things that make it difficult for an llm to learn.
Joking of course. Not sure copywriting code ever survives the AI era.
RMS fans advocating for copyright enforcement... what?
Essentially you're saying: AI companies are allowed to steal from all of us in order to make money, and in return they are allowed to read our private conversations and train future models in order to make even more money.
Yes and, furthermore, all property is theft.
This is like saying programs should be open sourced because binary is machine generated.
Llms are the compiler, and there are skill levels with vibe coding.
Show us the math proof you have under your pillow. If there is not any, you do not have right to anything.
What’s the point, if you can generate infinte code?
couldn't you always generate infinite code? just write more of it. but what open source lets you do is read the code for a particular piece of software, to see what it actually does. generating infinite code has nothing to do with that.
you could also do this before, but only with infinite time. open source value is shifting. open source is human-validated code. just OSSing generated code has no value.
If humans are directing AI to generate code, it cannot be open source unless the human decides to make it. It is their vision that AI acted upon. So, i dont see the logic in this post or thinking. this sounds like an another altruistic progressive california thing
It's hard to respond to something so reductive. This is the "everything's already been invented" angle.
Maybe by the same logic all code should be open source before AI if the author used stackoverflow, read a book, remembered their college education or read a blog?
Until a couple of years ago, learning was not considered theft.
Well good news, I guess. AI generated code is public domain. It cannot be copyrighted.
if i pay for AI to generate code, shouldn't i decide what to do with it?
by that logic, anyone who learned to code from open source projects should make their own code open source too.
If you put your code online just assume it's gonna get stolen and you'll never see a penny
You mean copied? I assume the purpose of putting it online is so it can be copied
Good idea. We just have to pay for the human-written part. Same with medical research and new medicines, new energy technologies, robotics, movies and music, and future AI. Eventually, as AI does more and more, everything will be free and open source.
You've figured out a way to eliminate money.
This article makes no sense. None of the arguments are sound and conclusive. First argument: Non sequitur. Programmers were also trained by open source software and from stack overflow and stack exchange. That's not a sound reason for their programs to be open source. Second argument: AI-generated content is not copyrightable. If that's the case, then it cannot be open source since open source licenses rely on copyright for their enforcement. The author confuses public domain with open source. Public domain and open source are two completely different types of categories with different legal implications. Third argument: This contradicts the second argument. Now programmers using AI should publish their code under an open source license or under CC0. That's a demand. Not everybody will do that and no argument is presented why they should do so. Fourth point: Not all code is a hundred percent human written and a hundred percent AI written. The author concludes from this problem that "Code like this, where humans put little effort, except writing prompts, should definitely be Open Source." There is no argument here at all. It's another personal demand without justification.
At least, the author should try to back up their demands with further justifications and make them less contradictory.
I mean, public domain code which is available is certainly open source. Just not the most common kind.
You cannot enforce that it's available without copyright and a corresponding license. I thought the author meant the code produced by AI should also be available to others somehow but perhaps I've misunderstood.
WOKE, no other word fits. You should move to another planet. Everything is stolen online, from marketing copy to books. And code is the one fight you want to have, the thing actually that is productive.
I’ve been thinking a lot lately about how LLMs have essentially made all code open source in a strange and very capitalist way: They’ve appropriated our human work—without regard for any licenses—by force and without any compensation to the authors, and are now selling the code back to anyone who’s willing to pay for it.
But yeah, aside from that, I’m also releasing all my Vibe Coding games under the MIT license: https://github.com/retrogames23/
Are they selling it back? All the best models are free downloads. At most you're paying for compute.
This may be the path to the true public domain.
Following this logic, there should be no fees at all on coding models, since they scraped the web on this knowledge. Same applies for books, pictures, music btw. This debate has not even started yet.
I see you and raise.. AI should be free
I think copyright law has simply failed to keep up with the reality of technological progress.
I don’t really see a problem with LLMs slurping up and regurgitating the world’s knowledge, if it was all freely and equitably available, and more importantly had been beforehand. We’d all herald it as an incredible innovation. And in some ways it is even now, because I do have access to it in numerous forms.
The problem is that some author’s livelihood is adversely impacted by this, as copyright is (in a sense) what ensured their remuneration for the original work. So LLMs undermine the existing social and financial order in deleterious ways. It’s the same thing we’ve seen play out in mathematics, a bunch if tech bros show up with a powerful tool, use it in a brash manner that likely causes real harm, then worse, start pulling up the ladder when big media conglomerates complain.
I suspect we’ll soon be in a situation where we no longer have easy access to unfiltered LLM outputs. If your OSS project was in the training corpus it will still retrieve it. It will not, however, retrieve content from NYT or other protected sources. The end result is that small-scale, solo creators are damaged, and large-scale creators are still afforded copyright protection.
There is no way to enforce opennness. The LLM gives me output, I iterate on it by hand and with other language models, who does it belong to now? We are ushering in a post scarcity world for a select few.
Nice pitchfork but I've got a hot take of my own:
No one goes to an LLM and says, 'Make code'. More often than not (and with exceptions) there's a very specific problem getting solved. The distinction between human 'effort' (whatever that means) and Ai effort (??) is more or less arbitrary... Infact, it's almost xenophobic and snooty. Pre-Ai humans had other humans build something like x but with y specific features. If it wasn't a problem then, it's not a problem now just because people don't have to directly deal with another human. In the same vein, I think people are forgetting that sometimes projects are birthed by slapping together code (or an instruction for code) from all across the Internet.
So really... What even is 'purely Ai generated' code? Nothing comes from nothingness and generally I'd prefer the purest expression of my ideas be protected regardless of who wrote it... Human4hire or Chat.
EDIT: This article is unnecessarily upsetting. I don't even understand the problem when at the end of the day. A shit ton of the value accrues to all of us... humans! Not even the companies (atleast not so far). If Artificial Intelligence really is a utility, like electricity, then wtf is exactly the problem? I hope nobody really gives any real thoughts to policing Ai usage... It'd be as unreasonable as policing a calculator or pen and paper.
Manners force me to stop here.
Code has lost almost all of it's value, who cares if it is open source. In a couple of years, you will say "make me a web browser" and 4 hours later performant, fully standards compliant web browser will be running.
The argument between open and closed source has little value anymore and will soon not even make sense to discuss. It will be like writing "2 + 2" and arguing over whether my "closed source" copy of "2 + 2" vs your "open source" copy of "2 + 2". When something is trivial to reproduce it becomes absurd to argue over who owns it.
And everything that can be handled as a common should.
That is everything bar one own mind and body, I guess, but your lineage might vary, and for sure some people out there would rather be deemed owner of everything under the sun, starting with human corps and souls.
Yes.
Until and unless every creator and contributor of every piece of training data consumed by these models is fairly compensated for the use of their works, then absolutely everything created by these models should be 100% open source - including the models themselves.
It is disappointing to see many people who espoused that copyright is bullshit now saying copyright should make people do what they personally want. I suppose it was never about freedom for them, only getting what they want when they want it.
Whether programs written by LLMs should be protected by copyright law is also an entirely different question than should the authors be legally required to release the source code under a specific kind of license.
All LLM code should belong to the entity that hosts the model.
This article misunderstands copyright law -- citing an example that argues against it.
It cited Naruto v Slater, which unsuccessfully argued that the copyright for a monkey's selfie belong to the monkey, not to the owner of photography equipment.
The courts dismissed the case as animals cannot hold copyright.
---
While the courts did not rule it, the apparent copyright holder is the photographer Slater who received royalties.
This is directly analogous to AI code, in which a human did not directly author the characters but regardless had "significant creative contributions."
Is this for real? If you get your way gosh knows the amount of "proprietary" software that would be open-source. Also really by this logic, even 5 years ago if I copied a method and made it my own from stack overflow or got an answer from there my code should have been open source. Kinda silly no?
All code should be open source.
At the end of the day you don't actually need to see the source code if you can just regenerate it yourself. If you use a later ai model, your code will probably be better anyway.
I agree
My patterns for loading data into Python is pretty much the same everytime with polars. I didn’t create that - I looked it up from a book somewhere. Do I need to open source it?
The AI machine just went out and read a bunch of examples then adapted them, which is what lots of programmers do.
If AI generated code is not copyrightable, releasing it under an open source license is meaningless.
Under those conditions it would be un-licensable (since a license is essentially an agreement not to sue for copyright infringement) but it can still be made open by publication. I'm skeptical that the "anything that comes out of AI can't be copyrighted" argument will end up holding, but in the meantime it provides a strong incentive not to make that code public.
Releasing it for sure is meaningful
I'm not sure you understand all the things that an OSS license does. For example: liability.
Maybe not, could you explain?
If AI-generated code should be open-source, then if an AI generates the code for Pokemon red, does that mean that it becomes open-source? I agree with the high level idea, but the way it is stated is too problematic.
Here here!
Honestly, I'm cool with that if all AI models are also made open-weight.
Open source people have had half a century to figure out how to make money with fairly little success (aside from a dozen or so very notable projects, and looking at all my AI slop, I don't see one of these hiding in there).
I don't particularly care about copyright or whatever ethical arguments yo may come up with -- I write code mostly for money, as do my competitors -- if you can figure out a way to stop them from using AI, I'm fine with not using it either. I'm fairly skeptical you can close Pandora's box though.
That aside, we may see more software going in a source-available direction -- either that or full cloud. Up until now being closed source was a competitive advantage, but now it's trivial to clone even fairly huge software suites, so that's less of a factor.
Having the source may also become a bigger advantage for regular users than before -- given they can now vibecode extensions/plugins, or customize the software more than what a config would personally allow. this probably wings the calculus away from a Photoshop-style license, and closer an Unreal Engine-style license.
Either that; or everything goes full cloud of course.
Hot take indeed, and something I disagree with. If I am paying for AI, my code is my code, no?
Yes, and you take ownership over it.
Well it is certainly not covered by copyright and to the extent it could be covered by trade secret is perhaps a debate.
This will never happen because it would essentially destroy every corporation now, the economy, and therefore every politician's funding source.
The output of machine intelligence has to be available to everyone or we're just breeding a new demigod class. We should make it a public utility, like water or electricity.
I can get behind that.
I think the legal standard currently in the United States is that all AI output is public domain.
This means that all AI generated code is public domain at the point of it being generated.
Of course just because something is public domain that doesnt mean you have to release the code publicly. Public domain just affects what others can do with the output. It doesnt mean you have to release the code yourself.
The new Epic Games Store is apparently entirely vibe coded. So that technically would mean that the source code for it would be public domain right? But people still can't copy the store or do whatever they want as you have trademarks protecting it. And Epic is under no obligation to release the code..
I am not a lawyer. It's possible I am completely misunderstanding the situation. Is there something I am getting wrong?
> I think the legal standard currently in the United States is that all AI output is public domain.
This is definitely not true.
The only thing a court has ruled is that the AI itself cannot hold any copyrights. Not that there are none or that using LLM based tools removes all copyright.
>First of all, AI models were trained on a massive corpus of Open Source code written since RMS announced the GNU Project and the Free Software Movement. That code was simply scraped from the internet and used without permission or proper attribution. The models were also trained on Stack Overflow and Stack Exchange sister projects, which are released under Creative Commons licenses that legally require attribution. Turning this collective knowledge into proprietary, closed-source code feels like license laundering.
Strong argument, but the case that such a strong transformation should maintain the original licence only applies to my knowledge to GPL, which consists of a very small portion of the training set. Creative Commons would not even require the source code to be distributed, just attribution, which may already be the case by the nature of the models.
>"The second reason is legal: purely AI-generated code is not written by a human, meaning it should not be copyrightable. The foundation for this principle in the US goes back to the famous Monkey Selfie dispute, which established that non-humans cannot hold copyrights."
But no one claims that the AI holds the copyright, unlike Greenpeace which claimed that Naruto the monkey held the copyright.
Why do people think that people receive the gift of open source software are somehow obligated as a result of receiving that gift?
Yes, it was trained on free software. Why does that mean my privacy should be invaded now if I distribute binaries I made?
Because instead of treating it as the gift it is, you're treating it as a service you are entitled to
I see you asserting that, I don't see him saying it...
Privacy?
The privacy of the source code files on my disk that I made, same as any other private files on my computer that I made that I choose not to publish.
There is still substantial human effort and R&D that goes into developing a complex code base with AI agents.
Downvoting sentiments that don't agree with the author's position is gross.
other copyrightable things aren't copyrightable if they're ai generated. why is code different? for clarity, i'm not at all being rhetorical, i don't know the answer to this question and i'd love some education if you're handing it out.
And AIs should be under libraries as AIs are the repository of all human knowledge scraped up without permission. Libraries are societies repositories/solution for this sort of access/use of shared knowledge like that used to create AI.
I think this misses the point a bit. All code should be open source. Full stop.
Have you written anything of substance? You wouldn't say such a thing if so.
Yes. I've had my career in tech for several decades and worked at both start ups and big tech. Currently running my own company.
agreed. The problem is _which_ open source, because what we're probably going to slowly realize is most software will now be a public good and proper governance is needed thats indistinguishable from a public utility.
But we're not headed in that direction are we.
I strongly agree. Look at what we’ve done here:
1. Software had a strong movement of freely sharing all code because they believed this technology was too important to be controlled by corporations.
2. AI companies managed to create millions of RL environments from GitHub PR’s and similar, and train their model to learn what it means to code from all the human labour in the world from the past 30 or so years.
3. Companies now use these closed source AI models, to automate and fire their human workers who enabled this.
Something has gone seriously wrong here, whether this falls under copyright or not is besides the point.
We weaponized a fundamentally good instinct that generated a lot of value for society and allowed corporations to rent seek on all the data these humans generated to help other humans not trillion dollar corporations.
“Well we’ll give them a UBI check”, the AI company says which is a complete non-sequitur. It’s not about the money, most of us have saved enough.
The only solution that sounds good to me, is a model trained on Open Source Data must be Open Source and all the code it generates must also be open source. This is the only way to stop the coming rent seeking. Even mathematicians will be happy with this change!
I agree with you on most things but man
> It’s not about the money, most of us have saved enough.
This is incredibly detached from reality. Most software engineers care more about money than they do about abstract notions of software from. Most certainly have not saved enough.
Fair enough, Im talking about Bay Area software engineers, most of whom have saved enough. Maybe not enough to live in the Bay Area whose house prices keep increasing with no end in sight, but plenty for elsewhere in the US (or the world) for that matter.
Even if you modify it before using it, it's still based on open source (often GPL) code, so just everything based in part on LLM output should be open source.
The author needs to realize that this is unenforceable.
In fact I'd argue that free software and open source is economically dead and unsustainable because of AI generated code and almost no one wants to pay for their tools because an AI can generate the missing feature for them close to $0.
Maintaining it is another thing, but assuming that the cost of tokens get cheaper, why wait for open source developers to maintain it when a frontier AI model can do it for far cheaper and quicker?
I am just saying what the average manager turned vibe coder would say who will refuse to donate or pay for established open source projects that they use for free.
Is it unenforceable to check that a company openly all in on AI is releasing all the source code of their products?
They don't have to release anything and most of these companies don't care about openly violating the license terms even if they are caught.
Unless the license holder has a lot of money for litigation, it is unenforceable.
They don't "have" because they have more money?
You're free to take that bet, we'll see.
> They don't "have" because they have more money?
My point is, if you don't have a lot of money to start litigating against the violations, the opportunists don't care and will continue to trample all over your "free and open source" software.
It not enforceable without a lot of money and the opportunists know that.
> You're free to take that bet, we'll see.
There is nothing to bet here. Truth Social is a clone of Mastodon (AGPLv3) and all they did was disclose the vanilla source code and not the modifications which that is required by the license. [0]
The same happened with Revolt (AGPLv3) and Andrew Tate's platform [1] and they never released any of the source code at all, nor could the developer enforce anything. [1]
[0] https://help.truthsocial.com/legal/open-source/
[1] https://insrt.uk/post/andrew-tate-stealing-software-revolt/
it’s not a hot take, it’s something most people wont bother to read cause it’ll never happen.
the only merit or such claim is to maybe move the overton window a tiny bit. maybe that’s worth the blog post
We already have a growing discoverability problem in many areas of content and throwing more slop into the mix is not the solution. This place for example has become a dumping ground for people, usually with no other activity here, to advertise their shovelware AI projects.
Here is a different angle. A programming language is a material like wood, aluminum, or stone. If you know the syntax and properties of it can you start trying to bend it to your will - you can make a ship, a satelite, or a children’s toy wihh tv the same language. You first have to spend years mastering the material and its limitations before you can bend it, so only a small number of the people who want to make stuff get to. Some people release open versions of a ship and a satelite you can start from, to lower the threshold of making stuff. They protect them to make sure giant shipbuilder doesn’t go making an army of their ships to kill their trade.
AI comes along and now grandma, who was formerly a midwife and teenager from home can start bending aluminum and carving wood with their words through AI. There are a bunch of open ships and satellites in the market, but Grandma is using her midwifing skills to ask AI to bend a new tool for her that helps turn breach babies around without requiring a c-section. No open source breach tools or medical tools on the market Grandma just needs aluminum and wood combination she would never be able to achieve without AI. She hammers down a lot - sending 1000 follow-up prompts to make the AI do stuff nobody has done before. Next her grandson decided to combine tungsten and wood to make a new toy that helps kids go airborne and land safely every time - like an egg-drop but for real life people. No open source code for that out there - he used the AI which knew the properties and the languages needed to get his idea to work. He hammers the AI with follow up questions and edits a bunch of the code where the AI defaults to the wrong setup, by trained convention. But he succeede. By this plan should they be allowed to copyright their work? Should their work be automatically open source so they can never monetize it either?
I will be the first to make all my work free and open-source once I can obtain air, clean water, basic food, shelter for myself and my family, electricity, heating, medicine, education and internet access for free.
Until then, I’m sorry, but I have to feed my family while we all navigate this goddamn global transformation process.
These don't seem related? I don't think the author says you can't charge or have a business model or make money. The code being open source is unrelated to these things.
Can someone help me understand who is still writing code by hand? Even though it’s been only 7 months since I stopped, it seems like such an outlandish concept to me. Can’t imagine opening up neovim and typing “const sm = …” nowadays.
I keep seeing posts like this (especially on reddit) that infer some people are still writing code by hand and its confusing.
There are people who actually like writing code and understanding things. A subset of these people who really like understanding things are called hackers which happens to be (supposedly) the target audience of this platform.
You can’t wrap your head around different people 1) adopting a new technology at a different pace than you or 2) enjoying different aspects of their job than you do?
Some people jump out of planes for fun. I can’t, personally, fathom doing that myself. But you still need to respect that different people behave differently (well, you don’t need to - we may be different in that respect)
Articles like this promotes extreme views while we should be moderate on what should be OpenSource or not.
First of all, nobody must force OpenSource. Period. Second, such exterme views spawn extreme views in the opposite side.
Discussion is good, but this article is too draconian to be practical.
I don't see extreme views in this article. When LLMs take creative content from copyleft code, the license says the output has to be copyleft. Otherwise, it's theft.
If LLMs weren't trained on AGPL/GPL/LGPL code, it'd be different. But they are.
They were trained on those lefty licenses.
Say you read those codes yourself and built same thing yourself but different code base. That's what AI is doing. Do you still enforce that open sourceness to those?
The current trend is for model output to be public domain, which is “more extreme”. Merely copyleft is not as extreme.
Trend don't mean shit unless enforced. You are free not to open-source it and got a choice.
- e.g. People use tax loopholes because certain areas are not enforced. The ever-true trend is to never use "immoral" loophoes but people still do.
The article claims the strict enforcement of making AI gen code as open-source, which is exterme AF.
Taking all of available human knowledge, knowledge that was published under limited licences, and commercializing all that already is a fairly extreme action. Merely arguing that perhaps the fruits of that harvest should also be available to the public doesn't seem that extreme.
IMO it'd be better if the weights themselves were considered public domain.
That’s not an extreme view. Anyone who uses software under a copyleft license should also release any derivatives under the same license. While this isn’t currently the case legally since the GPL was never written with the possibility in mind that LLMs might be trained on it in the future it still aligns with the spirit of the license.
>First of all, nobody must force OpenSource.
No, GPL and copyleft licenses very much do and should force OpenSource by its nature. If someone uses GPL or AGPL and doesn't open source, they should be forced to Open Source.
Why do you think people are now getting around these lefty licenses by AI-reverse engineering code now?
> such exterme views spawn extreme views in the opposite side
The opposite side had exrtreme views for at least 4 decades. They publish stuff, cram it into your face with advertisement and demand purchasing a license for consuming it. And if you don't buy they want you to pay thousands of dollars or go to jail. That's as extreme as it goes. Eff them. Reverse engineer, clone and open source everything. There shouldn't be a single proprietary piece of published software left in use.
This means “all software should be Open Source unless written fully by humans”, which practically means almost all software in the near future, which is a terrible idea for multiple reasons.
Or practically means that much less software should be written with AI
You say “should” which is an opinion. Legally you can’t enforce people to open source their software or never use AI.
Well, it's to be seen if you can't.
No you literally can't. Go outside a bit.
Great argument.
No, but all models trained on public data should be open weights by law.
What is public data?
This is like saying: All humans who learned from open source code should only produce open source code.
Humans aren‘t technology.
But we are. The finest technology we know of.