isn’t this a bit like saying “google drive is being used to easily pirate movies”?
Except it’s even a bit more ridiculous than that because they can technically just scan the files and the metadata as where these are entire models that are not developed or being deployed by hugging face, they’re just downloads hosted… I’m actually not sure what exactly they’re supposed to do about this besides not allow image models I guess? Which I frankly wouldn’t be against
i don’t think the distinction “image model” really holds anymore. most big language models can generate images on their own if you bully them enough. they’ve ingested enough binary data to know what a valid jpeg looks like and you can use that to get base64-encoded images out.
as for banning diffusion models rather than transformers… well we have text diffusion models now.
You think the model I use knows what a JPEG is? It just wants a picture of a gatdang hot dog
The image generator and llm are separate models
are you talking image diffusion models now? because CLIP isn’t really an llm in the normal sense. it’s just a classifier.
I hadn’t heard of that method.
can you show some examples of such images?
there are some in this paper: https://arxiv.org/html/2408.08459v2
It’s more like saying GitHub is responsible for piracy because they host codebases for torrent clients
No. One is about creating “content” and the other is about hosting “content.”
but… huggingface is a file hosting service.
It has specificity, it’s a niche. The niche is that of collaboration and aid in the pursuit of AI slop production, including the really horrible shit.
okay, but you see my point though, right?
Your point is that you’re trapped in semantics and you don’t comprehend what it means to deal with or combat networks of things.
no, my point is that attacking a file service for a problem caused by the ai giants is at best misguided and at worst an attempt to distract from what said giants are doing, namely creating the files in the first place.
It’s hard for me to figure out what HF as a platform can do here. What specific signal can be vetted to prevent this? Since models are non-deterministic I don’t know if keyword filters would even work.
To me this is different than Grok because Grok is a tool embedded in Xitter, that grabs the photos posted on the same platform to create the nasty images, so because the platform and tool are working in concert, X and Grok should be shut down over it if that function can’t be disabled.
We always have this misconception about what “non-deterministic” means. It doesn’t really mean what most people here think it does… And machine learning algorithms aren’t really non-deterministic in the way people think they are.
For example, you’d train an image classifier to categorize pictures correctly. If there’s a cat on it, it kinda needs to return “cat”. It’s not like they roll dice and deliberately return “cauliflower” instead. It doesn’t always do it correctly… I think that’s what you were trying to say with the non-determinism?! But we have classifiers who do it correctly in 99 point something % of cases…
So the way I think it could work is: HuggingFace uses Gradio to render those “spaces”. So every time an image element is rendered, it could pipe it through an image classifier and see whether there’s nude female breasts in the picture. If it “thinks” there are, discard the image and don’t return it to the user… It’d use a lot of compute, so be expensive. And probably come with a lot of false positives. But it’s not like it’s impossible to do these things. That’s usually how these safety/censorship measurements (“guardrails”) are set up.
The false positive from non-determinsm would appear, if you provide instructions “don’t give me a cat, ever!” it will return a cat sometimes if it is related in the model’s weights.
But yes perhaps the platform can run a filter that rejects nsfw output for any model that is being run as a demonstration on their servers, but it can’t really prevent people downloading a model and setting up an environment that makes nude image edits themselves.
Hmmh. A classifier doesn’t really have any instructions. Just an input (the image) and an output (the predicted category label). So that’s not really an issue, I think.
Downloading and running the model someplace else obviously is. I always like to think of software as a tool. It’s practically impossible to design an axe in a way it can’t do harm to anyone. Same with computers. And I don’t even know if there’s a point in trying. For liberal people, it’d probably be fine to offer dangerous items like an axe, gun or AI model, knowing it could hypothetically be used for nefarious purposes.
I think the red line is doing it, after being well aware the person is planning to commit a crime with the gun. That’s kind of unethical?! And maybe we should try to avoid handing out axes or random car keys to the town drunk or a clumsy 5 year old, as well. I think that’s pretty much where service providers are at. Once it’s a service, they have all the data… What prompt goes in and what image comes out, it all happens on their servers. And they know perfectly well what’s going on and who’s doing it. I think that’s the point it becomes some sort of obligation for them to care. In theory, at least.
But HuggingFace is -by far- not amongst the worst offenders. I believe most of the bad stuff happens on other websites. And there’s dedicated services for AI pr0n without any censorship. So I doubt they’ll have an impact in the broad picture.
AI output on images need a content regulator? Like an independent AI screening tool similar to material safety testing agencies?
Has to pass multiple passes but then we are just accepting the tech burden of this technology
it’s the in-band signalling problem again. how do you distinguish input and configuration when it all goes into the same pipe?
Tf is hugging face now I don’t want to Google it
Hugging Face is the place where you go to download models you can host yourself.
Honestly… I have no idea how they expect hugging face to address this, once you’re hosting the model yourself, they can’t control your system prompt or scan what it’s outputting or anything of the sort. It’d also be pretty impractical to scan all of these models to prevent anything, some of these models are terabytes in size, and I think just about anything that can output an image can output a “sexualized” image to some degree.
Seems like a nothingburger headline, really, a bit like saying “VLC does nothing to prevent users from watching child porn”. It doesn’t, but I don’t know how you’d expect it to when it just serves a basic function that doesn’t really have anything to do with the illegal part.
It’s backdoor outrage that will be used to ban local models so only the billionaire class get to own them and charge for their use.
Definitely on my list of suspicions, with this article coming out so soon after Claude had to drop their prices for Fable dramatically, and with models like Kimi’s latest competing with it for much cheaper. Not to mention this coincides with the push to ban “Chinese models”, it feels like this could be part of a broader wave to block the competitive open source models from US markets by any means available.
I only know that they recently scraped all of GitHub for training data
Of course it is…
TL;DR
WhErE gUaRdRaIlS?!Guardrails bad, something something free market, something something rational actors, something something common sense.
I always explain this “nudification” process as…
You know how if you put black bars over someone in a bikini or speedos, and suddenly they look naked? You haven’t seen the person naked, but you’ve got a good idea what they look like naked because your brain filled in the gap. It’s exactly what the diffusion model is doing.
I’m not sure it’s realistic to stop the models doing it entirely given how they work. If you want it to fill in the background when you cut your ex out of a photo, it’s going to fill in any other gaps.
deleted by creator
If you do it in the checkout line, yeah
