Return of the Revenge of the Return of the Compression Schmuck

Well, folks, this one is going to be boring for most of you. But the
compression jackass, Jules “Julie-baby-dumbass” Gilbert has been repeatedly
bugging me. And today, he replied to my latest “fuck off and stop sending me
mail” with a long-winded proselytizing response in which he requested that I
be courteous and not publicize the discussion of his “work”.

Fuck that

The entire correspondence is below.

Before I get to it: the short summary of what Jules is doing
is munging a file in a way that makes it amenable to recompression. In
the classic sense, this is removing a trivial bit of information from the
file, which thus allows is to be re-compressed just a tiny bit
more.

The catch is, of course, you can’t un-compress it without re-introducing
the information that you removed. In his case, you need to repeatedly
un-compress, un-munge, un-compress, un-munge, and so on –
exactly the right number of times. From some of his emails, he
does this hundreds to thousands of times.

You can’t un-compress his stuff without knowing how many times the compressor
ran. And how many times is that? Well, gosh, to know that, you need to know
some extra information. Anyone want to take a bet on what the
relationship is between the amount of additional compression he can get, and
the number of repetitions of his system? Anyone?

Ok. On to the transcript.

On August 11, he sent me the following (which was already in
response to my previous “stop sending me this bullshit” message of a week
earlier):

Hi Mark,

I’m the guy you don’t believe.

Well I have a question for you… What can I expect once you do believe me? I know what I want. I want a personal favor. I don’t care about a retraction, no, what I want is for you to do yourself! a
favor.

I’m pretty confident that in the next few weeks you’ll read that I have demonstrated my program, not just once, but again and again, to several people. And the demos will be done under “severe” circumstances, two machines connected by a floppy, etc…

So now, my favor…

I want you to read the Bible. That’s it.

I know, we live in a world that is incredibly corrupt — as soon as I wrote about “wanting a personal favor” I thought — he’ll think, but no. I want you to read the Bible.

If you are agreeable to this I will arrange for people I don’t know, actual strangers, to be in touch with you to verify the facts surrounding the demonstrations. Also, if you have someone in New England you would to suggest to attend, by all means, suggest away.

I am afraid that I am not a fan of Google. They are gay-friendly and also they suppressed lot’s of stuff about Obama before he was nominated, and again, before he was elected. So, if you please, no one from Google is welcome. Yes, I know, I use their net-mail.

I don’t like MS either — I admit, I’m pretty much an iconoclast. (If you worked for MS I’d be even less receptive.)

Mark, I know, you think I’m a fraud. And it is true that I have not made million’s in my career as a CS researcher. For years I worked as a consultant, doing compiler work and AI, and I made some great stuff but though I made a few VC deal’s, I never made million’s at one time.

So several years ago I identified repeatable compression as an interesting work area and have taken my time. But now I am trying to harvest what I have.

I hope you are well. If you are not, just disregard this note. But if someday you want to write me I will always be open to whatever you want to tell me.

Your friend,

Jules Gilbert

I replied on the 11th:

First of all: you are *not* my friend. You are a despicable schmuck who has no idea of what the fuck he’s talking about. I decide who my friends are, and regardless of what you think, you aren’t part of that group.

Second – What makes you think that I *haven’t* read the bible? I’m a religious Jew. My guess is that I’ve probably read the old testament in far greater detail than you have – and I’ve done it in the original language, not in translation.

Third – I don’t give a shit what kind of demo you can arrange. If you want to be taken seriously, you have to address the fundamental problem that makes what you claim impossible. Just suppose that you could use your system to compress things by *one byte*. Just one byte. If you could do that, you’d be mapping 256 inputs onto each compressed output. How, exactly, then, can you decompress an input? How does your system know *which* of those 256 inputs it should produce as the result of decompression? It’s very simple: If you can’t answer that – and I’m 100% sure that you can’t – then no demo matters. If you *can* answer that, then no demo is necessary. Answer that question, and I’ll admit that you’re right. I’ll even publicly apologize, and post regular updates on my progress re-reading the bible translation of your choice.

Fourth, I don’t think you’re a fraud. I think you’re an arrogant pig-ignorant idiot.

Finally – I can assure you that Google didn’t suppress *anything* about *anyone*. I can say that with absolute confidence – because I know about the technology we use, and actually picking specific information to exclude from the search database is quite *impossible*. Quite literally, impossible. We don’t have the ability to do excludes. Just indexing the data, without doing any kind of semantic filtering, is pushing the absolute limits of our technology. Semantic filtering would slow it down significantly – and any slowdown would destroy our ability to index data. I happen to believe that my employers ethics are strong enough that we *wouldn’t* do that if we could; but even if I was wrong about that, the fact is, we *can’t*.

So – put up or shut up and fuck off.

-Mark

As usual, Jules in incapable of respecting even a simple request. Yesterday, he
sent me:

Wow! Wow again.

I am sorry that you don’t consider me to be your friend.

You’ve said several things that I want to respond to, but let me deal
with the simplest first, about Google.

Although years ago I was heavily involved in mechanical language
understanding (I used CLIPS and built something that was used by CSC
for several years; I am not current today nor have I been for more
than a decade. So here I have to accept what you say, and, reading
your remarks, I am pretty sure that you are being sincere. So,…

I stand corrected. I will get back to the person who told me this and
advise them to look for a different explanation.

Now, about my being a “despicable smuck”, well, no, I don’t think so.
And I suppose I should be offended but I already knew that you don’t
believe what I say.

The problem you have is that I actually do have a system that
compresses previously compressed files. I know, you don’t believe me.
But it’s true, I do. Now, the issue here, I think, it this: You
think my claim must be false, that such a claim is false on it’s face,
while I know that it is indeed possible, in fact I do it all the time
when I am on my R&D machine.

And that is why a demonstration, assuming two machines, a floppy
interchange, no wireless, no wires, no memory sticks, turning one
machine off while the other is on, careful examination of the machines
— all of this means that either I am a great magician or that I
actually have done what I claim I have done. And truth, I am not
faking, also I’m not good with slight of hand stuff.

You seem to assume that I am attempting to defraud people, that I
haven’t done what I say I’ve done. I recognize now that I am not
going to convince you unless I send you my source code, or, at the
very least, announce my methods. (Though I am glad that you don’t
think me a fraud. I’m not.)

Well, gosh, I’m sorry, but I can not simply ignore the commercial
implications and tell you how I do what I do.. I have investors, I
have a wife who wants, when I die, to continue to be able to eat and
pay the electric bill. So if you don’t mind I will keep my
intellectual property to myself until I extract value from what I have
done.

This is why I don’t patent — because the moment I do, gigantic corps,
such as MS, will take what I have, even ignoring right and wrong. And
you’re pretty knowledgeable, you know this is exactly what they did to
Stack Electronics. And, as I look at those events, I would say MS got
away with it, that the penalties they paid in cash and stock were
nothing compared to the degree of damage sustained by Stack.

Now, about the Bible. Perhaps I have seriously misjudged you. But if
you are telling me that you are trying to be a good Jew, other than to
say that you are rare, I need to apologize to you and not trouble you.
I believe that things are soon going to get very bad for Jews in
Israel, and I notice their level of isolation is increasing. For
decades, because the US was a Christian nation, Israel received a
great deal of help from us. But I am doubtful that Obama and others
(Hillary Clinton, for example,) will help Israel the next time she
needs US assistance. As I see Obama, I see someone who is completely
godless and who has no intention of helping Jews.

But forget this — look, he won’t be able to run again, too many
people will be clamoring for proof of his citizenship. And I think
he’ll try to stay in office. How?, I have no idea… So I am
expecting real trouble ahead and one thing for sure, it’s important to
be on good terms with the God of the Bible.

I did know that you are Jewish, but the nature and kind of interaction
we’ve had led me to see you as a very different kind of person. None
the less, I do not want to offend you. Not one bit! I am a Jew, too,
at least by birth, but growing up (single parent,) I was not brought
up as a Jew, rather the religion of our household was atheism. It was
as an adult I looked around and decided that Jesus is the Messiah.

Consider Abraham and how, when he and Lot ran into trouble because of
the size of their flocks, how Abraham decided to split but graciously
allowed Lot to take the better land for his group, while Abraham and
his fellows made do with less good pasture lands.

My point: You just called me a “despicable smuck”, and I guess I
don’t associate such behavior with the kind of person Abraham was/is.
However, I can be wrong. And while I don’t agree with your
assessment, if you are telling me that you are a Jew and that you are
trying to be a good Jew, I need to respect that. If you do decide you
want to talk, say so.

As to:

> you have to address the fundamental problem that
makes what you claim impossible.

and also:

> Just suppose that you could use your system
> to compress things by *one byte*. Just one byte. If you could do that, you'd
> be mapping 256 inputs onto each compressed output. How, exactly, then, can
> you decompress an input? How does your system know *which* of those 256
> inputs it should produce as the result of decompression? It's very simple:

I have an answer. It’s just not the kind of answer you want.

First, I can not store more than 256 states in one byte. I hope that
I have never said anything that led people to think I thought that I
thought I could. I have said, in jest, that I use the space between
the bits, stuff like that, but I assumed that no one took me
seriously.

However, this DOES NOT relate to the core issue. The core issue is:
Can a previously compressed file be re-compressed. And like it or
not, Mark, even assuming the prior use of a good quality compressor,
the answer is “yes”, at least that’s the answer until the file/message
is pretty small.

I know, you don’t believe me.

Now, about being added to your friend’s list. If you have any
deletions, please consider adding my name. (How else can I say this?,
I’m trying to be your friend.) I suppose I can tell you that if
someday I patent I will make sure to notify you. But this is not
likely, that’s publishing, and invites violations, my only response in
such cases is to sue, and frankly my physical health isn’t so good, my
wife has mental health issues (now she’s fine,) and I don’t want to
spend years in and out of court.

I repeat, if you decide that you would like me as a friend, please do say so.

I can leave you with this, I have a friend who runs’ a conservative
website. He believes that Google did suppress information that wasn’t
favorable to Obama. I’ll tell him that someone I trust believes that
such actions could not have happened. I will not mention your name or
involve you in any way that could come back to you.

–jg

And, once again, my reply:

No one would argue that you can’t munge files in a way that allows you to recompress them. The question is, can you *uncompress* them afterwards?

And the answer is, no, you can’t.

“Repeated compression” like you’re doing is very simple: just throw away information. Less information, and presto, smaller files. And you can do that over and over and over again, as often as you want, until you get the files as small as you want.

But if you can’t decompress it afterwards, what good is it?

I can come up with a half dozen schemes by which you can take a compressed file, fiddle with it, and then compress it again. They range from the profoundly idiotic to the surprisingly subtle. But they all have the property that the way that they make the file compressible is by reducing the information content – they’re discarding information. And if you’ve discarded any information, you can’t decompress it.

And I think that on some level, you realize that. That’s why you wouldn’t actually answer my question. But that’s the key. If I give you a file that’s been repeatedly compressed by your system – the compressed file and nothing else – can you return the original file to me?

That question is just another way of phrasing what I asked you last time. If you could compress any file N bytes long into a file N-1 bytes long, that would mean that your “compression” process mapped 256 inputs onto one output. So how can you uncompress? How can you know which of those 256 files should be the result of your decompression?

And if you can’t answer that question for a process that would only reduce things by one byte, then how are you going to answer it for your system, where you claim to be able to map gigabytes down a tenth of a megabyte? You’re talking about a system where for any compressed file, there will be billions of possible inputs that produce one compressed output. How are you going to figure out which one is the correct result of un-compressing?

As for your self-defense: you *are* a despicable schmuck. You’ve ignored my repeated attempts to get you to leave me alone. You insist on repeatedly attempting to preach at me despite the fact that I have made it abundantly clear that I am not interested. You have shown nothing but disrespect and contempt for my wishes. Do you think that makes you a good person? Does your precious false messiah tell you that you’re really a great guy for doing it?

You’ve made slanderous claims about a company full of good people. You’ve made slanderous claims against the Jewish community. You’ve made slanderous claims about just about everyone who doesn’t share your own petty stupid ideas. Do you think that *those* things make you some kind of wonderful person?

You are a despicable, vile, hateful person with no respect for anyone but yourself. And you’re a pig-ignorant moron.

Do me a favor, and listen this time: fuck off.

Did he listen? No. This morning, in my mailbox:

I get it. You don’t want to talk.

Okay. I will make this as brief as I possibly can and I won’t discuss
any ancillary issues.

You recently offered to read the Bible, posting notes regarding your
progress online in exchange for my disclosing how I do what I do.

Unfortunately (as I explained,) doing this would make it impossible
for me to profit from my work. (I’ve got about 1/3 time, over a
fifteen year period invested. Five years.) So I can’t accommodate
you.

I am offering you a modification:

a) You don’t have to post your reading progress. In fact I thought
your offer to post unnecessary. I trust you, and although you have
made it plain that you are not my friend I am hoping to change that
and one thing I can do to help effect that change is to be your
friend, that means trusting you. So if you tell me you have reading
something, I will believe you.

b) For my part, I propose something similar. Yesterday I met with a
“money guy” — someone who is experienced at raising investment money.
Obviously he wanted to see the program, he understands that some
sharp people think that my claim has to be false, that it’s like
claiming perpetual motion or somesuch thing.

But not only did he insist on a demonstration, he further insisted
that everyone who was a candidate investor WOULD FIRST BE REQUIRED to
watch the demonstration. Not a video on YouTube or a simulation (I’ve
never faked a demo — that would be fraud, out and out — I’m a
researcher, what I do is real. That’s all I do. And a couple of my
investors, one particularly is a very accomplished C programmer, he
knows exactly what I do, he has compiled the code and run it.)

So here’s my modified offer; I will supply you the names of several
people, strangers, candidate investors, who have seen the demo (we
expect to start doing demo’s in October, the key guy is now arranging
for a hotel conference area for two days during the week for the month
of October.) So, if you are willing to accept, say a half dozen
people advising you they saw it — ie., to email you a very detailed
description, and I can ask one or two of them to phone you so that you
can ask questions, then you can be pretty certain that I have what I
say I have.

And once again, I respond in as hostile a manner as possible, hoping
to drive the stupid bastard away, so that he’ll stop spamming me:

You don’t fucking listen, do you?

As far as your super-duper compressor goes, as I keep saying to you: demos don’t matter. I don’t care whether you claim to be honest or not. I don’t care how many witnesses you have. None of that matters.

There’s one essential fundamental question about your system. If you can answer that one question – which I’ve asked you several times – I will absolutely accept that you’re able to do what you claim. I won’t need to see a demo. I won’t need to hear from witnesses about your demo. I won’t need to see your code. Answer that one question, and I’ll believe you. But if you *can’t* answer it, then you’re full of shit, you can’t do what you’re claiming, and no number of demos can possible change that.

Here’s the question, *again*, stated as clearly as I can possibly put it.

You claim to be able to compress *any* file down to something on the order of 40Kbytes by using your repeated compression system.

So let’s make the problem incredibly simple. Suppose I give you a set of files that each have 40,001 bytes, and your compression system can, somehow, compress all of those files to 40,000 bytes. At this point, it’s not particularly useful – but it makes the problem clear. If you can do this, that means that for *any* 40,000 byte compressed output, there are, on average, 256 possible inputs that could have been compressed to it.

If I give you a 40,000 byte compressed file, how can you determine which of the 256 possibilities is the correct result of decompressing it?

If you can’t, then what you have isn’t a compression system.

I can invent a “repeated compression system” that works wonderfully, if I don’t need to decompress. Just off the top of my head: break the input into 1024 bit chunks. For each 1024 bit chunk, compute the a 1024-bit SHA hashcode, and replace the original chunk with the computed hashcode. Then run it through your compressor again. Presto! It will, with near-100% probability, result in a smaller compressed file. And you can take the result of that, and repeat – and it will *still* likely result in a smaller compressed file. Evetually you’ll reach a fixed point – but you can easily take terabytes of data, and reduce them down to a couple hundred K at worst.

But it’s completely useless – because it’s not really compression. You can’t get back the uncompressed data. It’s gone.

That’s exactly what’s wrong with your system – unless you can explain to me, for my example, just how your decompressor knows which of those 256 outputs to generate.

If you can’t answer that question, then just go the fuck away and leave me alone. Don’t waste my time with your proposals, or with your inane babble about how you’re going to demo it, how you’ve got money people wanting to invest, or any of that crap. Answer the question, or shut the fuck up.

At last I’ve apparently made him angry, but alas, not angry enough for him
to be willing to stop pestering me.

Alright — now you are getting me angry, and although you and I have
been round and round on a couple of matters, until now, you haven’t
made me angry.

Mark, give me a couple of days and I will try to formulate a response.
For you. You haven’t promised me confidentiality but I hope you will
be a gentleman and not broadcast what you learn.

I don’t broadcast your remarks, please do me the same courtesy WITH
RESPECT TO compression issues.

I’m not sure this is wise, but I will assume that our conversation is
secure. (That’s a stretch, because of my work, I get people trying to
put worms, viruses, and other assorted animals on my little Apple
appliance.)

I hope Mark, that you and I do become friends. I don’t know if other
Christians have spoken with you about Jesus but my plan for you (if I
can get you to relax a little bit and be a little more gentle,) is not
to tell you all about Jesus or what you will read in the New Testament
(not that I think any of this is a bad idea, I’m just pretty sure that
at this moment it will get us nowhere.)

No, my plan is to ask you to read the Old Testament, such books as
Zechariah. As Psalms. And Isaiah. Here there is plenty of real
evidence as to identity of the Messiah — and besides, it’s God’s
“job”, not mine, to work inside your mind. He chooses who he wants
(yes, I think he wants you.) But my focus is to ask you to read the
writings of Abraham, of Moses, to read David and Solomon. And yes,
Isaiah and Zechariah.

God direct’s me. By the way, I think he directs you too. And
remember, he told us something, something very basic, something so
important that it transcends almost everything else.

It’s this: That God made us for fellowship with him. And as I
understand things, though I don’t speak Hebrew, I believe that’s what
he said, that he made us so we could be his friends.

In the Christian model, people who follow the Messiah are as small
cells in a body in which the Messiah is the head. By the way, Jesus
made this remark more than 1500 years before Louie Pasteur looked
through a microscope and discovered bacteria — which led to the
realization that we creatures are composed of cells.

And about this work thing, remember, even Adam before the fall was
given some work to do. God told him to tend the garden. Yes, God
made beaches and wants us to enjoy them, he also intends that we be
productive people. I mention this because some people (apparently,)
have no idea what to expect of God. Men such as Adam, Abraham, Moses,
Nehemiah, and (I think,) all the men of the Bible (the Patriarch’s,)
worked. Also Ruth and Naomi.

About my technical description…

I am not going to write anything that tells you how it’s possible to
squeeze 9 bits in a byte — if you really try and close your eyes.
(Now that’s a concept!!)

You know what I mean by “framing” how, in a discussion, it’s important
to frame the conversation so that both parties think what is being
said by the other person is useful. Are you with me?, well I messed
up very badly at the beginning of my internet career as a writer of
perpetual compression applications. (I went public with something
that was very weak and made just about every mistake I could possibly
make.)

Here is one very important thought, not merely my key thought — it’s
the foundation for everything I am doing.

My output files are only expected to produce, during decompression,
the prior output file. (ie., prior from the view point of
compression.) So a zlib file my program produces only has to produce
another file, yes, a zlib output file, somewhat bigger.

The last time this is done the just built file is a copy of the client
input file (if everything went well, I don’t provide checksums yet.)

Now, years ago I said this on the ‘net and everyone jumped all over
me, believing that I was lying. Nope.

I have to think very carefully about what I can say and what I need
not to discuss.

If I sign this “Your friend” please do not start cursing me. Okay?

Your friend,

–jg

PS: And yes Mark, I actually do read your email and consider what you
say very carefully.

44 thoughts on “Return of the Revenge of the Return of the Compression Schmuck

  1. proflikesubstance

    Dude, just ignore. It doesn’t matter how hostile you are, the response is all he wants. Add him to your spam list and forget about it.

    Reply
    1. Jac

      I agree with proflikesubstance. Don’t let the Blogstalker play power/control/attention games with you.

      I’m really enjoying the maths; if you can explain it to me, you can explain it to anyone. Great stuff.

      Reply
      1. Brian

        Indeed. Please note that, everybody. Jules Gilbert has his own entry in the comp.compression FAQ, and note further that this section was added in 1996.

        Over the last 14+ years, Jules Gilbert has heard every argument that is possible to make about why what he’s doing is impossible. And he has heard them all many many times. He doesn’t give a fuck what you say. He’s not just stupid and obnoxious. He is incorrigible. Unfixable. Relentless. You can write him emails until your fingers bleed, and nothing will change.

        Ever.

        You can argue that he’s not technically a troll, but he might as well be for all the difference it makes. Don’t feed Jules Gilbert.

        Reply
  2. Jens Fiederer

    Bad enough that YOU need to read all that preaching. I feel for you, I really do….but MUST you pass it on?

    Thank you for the courtesy of at least labeling them clearly so that I can ignore any further posts.

    Reply
  3. Eric

    You know the old one, right, about how you should never try to teach a pig to dance?

    You don’t accomplish anything, and it annoys the pig.

    I think that applies here. I’m not sure why you bother responding.

    Reply
    1. MarkCC

      I have a unfortunate tendency to enjoy provoking lunatics. It’s a nasty habit, but that’s sort of where the whole bad math side of this blog came from. So I sometimes egg them on, to see what kind of lunacy will come out, in the hopes of producing a perfect gem of lunacy.

      On the other hand, sometimes I let myself get drawn into something stupid like this.

      Reply
  4. Jarmo

    If you send 10 packages from 10 locations to 2 addresses without a return address, how are you able to send those 10 backages to those original 20 addresses?

    Reply
  5. S.C. Kavassalis

    Wow… I can’t believe you actually responded to him more than once. I have never had a crank try to push a nonsense theory at me while simultaneously proselytizing for their religion; that’s an intense level of clueless. Simply, baffling.

    Reply
  6. Bill

    I nominate James Randi to be your representative at the demo. I understand he’s pretty good at cutting through bullshit.

    Reply
  7. hawaii2000

    I learned a long time ago that there’s no winning in dealing with people like this. It’s like trying to converse with a dog. The dog has no way of understanding anything but simple commands like sit, speak, fetch, etc. Beyond that, the dog just sits there staring at you.

    When this guy writes back nonsense, he’s that dog staring at you.

    Reply
  8. Aron

    Seriously why bother responding? Why publicly make fun of him? Yes, old man is not completely sane but hes harmless, just let it go, you may very well end up senile one day too, should we all make fun of your ramblings then?

    Reply
    1. MarkCC

      If I wind up harassing innocent people, even after they repeatedly tell me to go away? Then yes, I’ll deserve to be made fun of.

      But I don’t believe this guy is insane. I think he’s stupid and obnoxious. In my book, that makes him a lot more culpable than someone who is genuinely crazy or senile. He’s an asshole, and yes, he absolutely deserves to be made fun of.

      Reply
  9. col

    Don’t feed the troll. He’s clearly unhinged and unlikely to respond to reasonable, or for that matter unreasonable, suggestions to FOAD. Just ignore him.

    Reply
  10. The Science Pundit

    It mentions Mike Goldman’s $5000 “Compression Challenge.” If Cranky McCrackpot really thinks he’ll win the $5000, I’m sure he’ll have no problem with Goldman’s $100 entry fee.

    I was wondering if this claim might qualify for Randi’s $1M challenge, but I didn’t realize that there was already a challenge out there specifically for this breed of crank. Boy is he several screws loose!

    Reply
  11. David

    next time he writes, ask him how Jesus managed the decompression thingy with the loaves and fishes.

    Reply
  12. Tualha

    Mark, don’t you know better than to feed trolls? Someone who works for Google should have been online long enough to have learned that basic rule. You can tell this idiot to go away until you’re blue in the face and he never will, any more than a spammer would.

    Reply
  13. efrique

    He refuses to engage with your underlying point because it would stop him continuing in his delusion.

    He doesn’t want to understand that your question is the single, fundamental question that anyone who claims to have a compression scheme that always works has to answer.

    I come across the same problems in creationists, flat earthers and a bunch of other loons.

    I point out (when people ask me why I debate creationists and some other kooks online when they’re never going to be convinced) that they’re not the only ones reading it. The same is true here – by doing it in a public forum, many people can benefit from your explanation. While I aleady understood that universal compression wouldn’t work, I now feel better able to get to the heart of the matter quickly.

    Reply
  14. bunny

    There’s a really easy interactive proof system to solve this problem to everyone’s satisfaction.

    1. Gilbert sends you a compiled binary of his decompressor. He can obfuscate this to his satisfaction, to prevent you or anyone else from stealing his work (or in nerdspeak, the proof is zero knowledge).

    2. Send Gilbert a several-gig file. Maybe a movie file, maybe just pipe /dev/urand into a file for a few hours. Probably include an md5 hash, so he doesn’t have an excuse if it fails.

    3. Gilbert sends you a <100KB file back (and its hash).

    4. After verifying the file, run the decompressor on it (probably in a VM without network access), and compare the output with the original file.

    Since he provided the decompressor before you gave him the file to compress, if his claim is not true, the probability that he can possibly provide a sub-100KB file that decompresses to the original file is equal to the ratio of the size of the compressed file over the size of the initial file (somewhere around 10^-5). However, if his claim is true, he will be able to with 100% probability produce a file that regenerates into the original.

    Gilbert has an advantage in this proof system (on the order of 1 in 10^5 chance of making you falsely accept), but I think you could probably grant him that given the mathematical impossibility of his claim.

    Reply
  15. Vicki

    He lies. He says he gets it that you don’t want to be bothered, and then he goes on at length, bothering you.

    Reply
  16. Michael

    Honestly I think you’re giving him precisely what he wants. He knows that all he has to do is get through a few insults from you and he’ll get his message out there. I’m convinced his message has nothing to do with compression; from reading through this I believe his end goal is to use your blog to get someone — anyone — to say “hmm… this guy seems intelligent so perhaps that Bible on my bookcase could actually be worth reading.”

    This approach is very counterproductive. Anyone of any religion who wishes to convert others should do so simply by being a good example. Being a crackpot and repeatedly sending the same nonsense to an unwilling recipient is simply stupid.

    May I suggest you just right click > mark as junk mail?

    Reply
  17. Nick Johnson

    @bunny : I think this is needlessly complicated. Simply send him a large, randomly generated file, and ask him to send back a single, smaller file – a tarball or an executable, whatever – that when executed recreates the original file. That is, require the sum of the size of the decompressor and the compressed file to be less than the original.

    Reply
      1. Nick Johnson

        That’s not really a problem, because he has to include the code to do the decompression in the submission. If he can consistently come up with a decompressor for any input that, bundled with the compressed data, is smaller than the original, then he’s still somehow worked around the pigeonhole principle!

        Reply
        1. James Sweet

          Only sort of. If, for a given input, he can come up with an algorithmic method for getting reasonably close to the output, and then just efficiently encode the difference (using off-the-shelf lossless compression)…

          If you require a reasonable amount of compression (say 10%) then in practice, this is probably impossible for truly random (not pseudo-random!) data of a sufficiently large size. But it’s still risky.

          If the challenge is to remove just one byte, it may be feasible to defeat it… although, making him include the executable as part of the tally might make it effectively impossible. (Though not theoretically impossible)

          Reply
        2. James Sweet

          Nevermind, I changed my mind — your challenge actually is perfect, both in theory and in practice.

          Obviously the “in theory” part works if you extend it to all possible inputs, because of the same 1-to-1 mapping argument.

          I had been concerned in theory about the “in practice” part — it will make sense in a second! — because I thought that, with enough ingenuity, there might be a large enough subset of possible inputs for which the “random occurrence of non-randomness” was frequent enough to make it compressible this way.

          But I’ve changed my mind. Assuming truly random (not psuedorandom) data: For smaller inputs, enough exploitable “non-randomness” might occur by chance to come up with a close algorithmic approximation of the data, but you’ll never get enough compression to balance out the executable data. And for larger inputs, the odds of there being a close algorithmic approximation are astronomically small.

          So your test would be quite effective. As long as the self-expanding executable isn’t allowed to connect to the internet of course! 🙂

          Reply
  18. CaptainBlack

    In the last of your “friend” ‘s posts he wrote:

    “My output files are only expected to produce, during decompression,
    the prior output file. (ie., prior from the view point of
    compression.) So a zlib file my program produces only has to produce
    another file, yes, a zlib output file, somewhat bigger.”

    Is this not a confession that the compression is lossy (allowing for ambiguity of language)?

    I read this to mean “the result of decompression has expectation equal to the input to the compression stage”. Whatever he might think this means, it is indicative of a lossy process and so if his compression step is repeated enough times the probability of being able to restore the original goes to zero.

    Or have I misread the original quoted fragment?

    CB

    Reply
  19. Tim

    What’s the betting his method “to squeeze 9 bits in a byte” involves telling them to Jesus? Decompression then becomes simple – just ask Jesus to read them back to you. Wait maybe I should patent that.

    Reply
    1. MarkCC

      Sorry, he’s not that straightforward.

      His trick is actually pretty simple. He does his munge and recompress, munge and recompress thousands upon thousands of times. And the only way that you can get the decompression result is by knowing how many times.

      That number of repetitions is an outside data source – an extra chunk of information that you need aside from the compressed file and the decompression executable. And depending on whether he always uses exactly the same munge operation, there’s possibly even more hidden information.

      It’s the usual shit with decompression cranks. Either they can’t decompress their files at all, or they need some back-channel extra information to make the files decompressable. Once you add in the back-channel data, you’ll find that there’s not really much extra decompression going on.

      To be clear, it’s entirely possible to find a way to improve on gzip type compression algorithms. They’re definitely not compressing to the minimum Kolmogorov complexity of an input. So it’s possible that you could take the result of a gzip, and do some non-lossy transformation of it that allowed gzip to take advantage of some redundancy that it didn’t catch the first time ’round. But you can’t do that iteratively – you can’t make it smaller that its Kolmogorov complexity.

      Reply
      1. James Sweet

        So yesterday I was thinking about this out-of-band information thing… and I think I could come up with a pretty cool hoax to make it look like I was meeting a “remove one byte from any file” challenge.

        The idea is this: I know some information about the person doing the judging and where/when they will be judging it. For instance, I might know they always try to decompress my files within a day or two of when I send it, I know their IP address or at least the first three numbers, or whatever.

        Then “somehow” (I haven’t figured this part out yet, but it seems like it should work) I use this out-of-band information as part of the compressed data. So when I compress a file, I give it the expected date and IP when/where it will be expanded, and I use that to remove data from the original file. If I could even come out a byte ahead…

        totally useless of course, but it would be cool if I could have Mark keep sending me gigantic files of random data, and keep sending back smaller compressed files that “seemed” to expand correctly. heh…

        Reply
  20. Tuomoђ

    Those trolls are fun when they turn their levels “on”. Someone would say that ansvering them is feeding the troll. Perhaps they are mad or senile, and perhaps it is “not nice” to tease and ridicule them anyway – even if they are “considerably healthy”.

    But in other hand they seem always so lonely and so full of spirit of debate and ridicule others. With that point it is perpahs right thing to talk with them. They feel not ignored. Their self respect grows through “enemy contact” – they started feel that they are “soldiers of truth” and not just “lonely miserable guy”. They gain feeling that they matter.

    But it is totally uncool to have your mailbox stuffed. They have always unbelievable amounts of bits stored. If you got two trolls at the same time, it is totally nightmare. And sometimes it just don’t help. I got one devotet christian who wanted to save me. – Havin a one different opnion in one detail and I just got straight “atheist” -label and anything can not change that one. Finally I got messages in my facebook. I blocked name from friendlist and all, but it just did not change. My pleadges with content “go away” did not do the job. Not even when I started to pack them with some curse words. Blocking did not work ; It is so easy to make a new mailbox today, so jus different address and here I got flood again.

    Trolls can be annoying when they gain confidence.

    Reply
  21. Ann

    > So, if you are willing to accept, say a half dozen people advising you they saw it

    This is such a great example of what happens when religious and scientific reasoning converge willy-nilly. The quote above refers to a compression algorithm, but it’s nearly identical to evangelical arguments for the reliability of the gospels.

    Reply
  22. Jordan Rastrick

    What a sad way for a gifted person to be spending their time – you could be working on whatever amazing new services Google have in the works, or posting about some elegant new data structure in Haskell, or even doing nothing at all which at least would be better for your blood pressure.

    Instead you are engaged in a long, angry correspondence with a deranged person, who’s fundamental disconnection from reality only becomes more evident the more he writes. You feel compelled to share it with the readers of your blog. You have repeatedly asked this person to “put up or shut up and fuck off”, when you know – with the highest degree of certainty a human mind can ever obtain – that he will never be able to “put up”, while you are quite capable of permanently shutting him up yourself with a single click of your mouse.

    Any possible value to this discussion, in helping others clearly understand the mathematics behind the impossibility of ideal compression, has already been completely realised by the previous post on this topic.

    If a homeless person in the street came up to you incoherently insisting that they were the Messiah, would you ignore them and walk away? Would you stop and try to calmly discuss the merits of their religious views? Or would you get irritated enough by their obstinate foolishness that you would argue loudly and publicly, and record the argument on your smart phone to post on YouTube, hoping that the possibility of public humiliation might get them thinking straight?

    I will simplify my suggested response from the last comment thread even further:

    “No offence, but I am permanently blocking your emails, and I suggest you seek the help of a psychiatrist.”

    Reply
  23. Timothy V Reeves

    Very, very frustrating reading; you just can’t corner him with logic can you?And when you do succeed in cornering him he plays the industrial espionage card and masquerades as the keeper of deep technical secrets. It reminds me of the so called “Searl Effect Generator” and the like; any attempt it check it out in the public domain meets a wall of intrigue and mystique, not to mention egos the size of planets that, I suspect, are out to protect themselves at all costs. I think Jules needs less a psychiatric shrink than he needs to shrink of his massive ego. But only he can do that.

    I found myself getting wound up just reading the above. I’ve had some problems with this sort of email myself so thanks for publishing it; I’m studying it carefully and considering my options.

    Reply
  24. Paul Murray

    “In the Christian model, people who follow the Messiah are as small cells in a body in which the Messiah is the head. By the way, Jesus made this remark more than 1500 years before Louie Pasteur looked through a microscope and discovered bacteria — which led to the realization that we creatures are composed of cells.”

    Jesus did not say cells. He said “I am the head, you are the body”, he said “I am the vine you are the branches”. Paul, I think, compared individual christians to “members” of the body. No cellular knowledge there. Just the notion that bodies are made out of parts, that have functions.

    It’s just typical, you know. These people do knot know what is actually in their holy books. I suggest that – if he’s so keen on Jesus as the jewish messiah – that as a study he go through the book of Matthew, locate every place where Matthew claims that this or that is a fulfilment of scripture, and then go study the scriptures that Matthew is claiming to have been fulfilled.

    Reply
    1. MarkCC Post author

      Why would I regret it? People file patents all the time. What harm does it do to me?

      More importantly, the fact remains that I’m right. Anyone can file a patent. And there are plenty of patents for things that don’t actually work. Filing a patent doesn’t prove that his system does anything useful.

      Look, there’s a reason that he’s always refused to respond to the central point of this argument.

      His “system” inevitably compresses multiple inputs to the same compressed file. There is no way to get around that – there are infinitely more possible inputs to his system than there are outputs. So how, given a “compressed” file, can he determine what the correct uncompressed form is? No matter how many times this is explained to him, no matter how many times he’s asked about this, he never comes up with a meaningful response. It’s a crucial issue – what good is a “compressor” that can’t correctly produce the un-compressed file?

      Reply

Leave a Reply