It is an interesting presentation, but I do NOT understand Katie's explanation about how they were going to minimize the bias [to "see" already predicted black hole visualization] while creatively interpreting inputs from sparsely placed telescopes around the earth.

Do you understand Katie's explanation?

Disclaimer: this is based on watching the talk and some basic machine learning knowledge. Im no expert.

They have a sparse set of data that is part of an image. They have trained a model to look at the sparse set and make an educated guess about what the full image looks like. They do this by feeding it full images.

The full images you feed into the model thus have an effect on the final image generated. In order to see how large that effect is, they trained different versions of the model with different sets of complete images. Some were images of what we thought a black hole looked like. There is potential that this heavily influences the model and ensures that the output looks like what we expect it to, even if that isnt actually true.

They also trained the model with non-blackhole images. Since the output of the model was approximately the same, this indicates that the resulting output picture doesnt look like what we think a black hole looks like just because it was trained with black hole images. It likely really looks like that.

The model doesn't need to be told what a black hole looks like. The sparse measurements combined with knowledge of how sparse data can be combined to form a generic image is enough. The model learned that the sparse data is not likely pure noise, instead there are shapes and lines and gradients that relate the sparse data points to each other.

Her analogy of sketch artists is good. If you have a functionally complete description and give it to 3 sketch artists from different cultures who are used to different looking people, they will still draw the same person. However if your description isnt actually detailed enough, their sketches will significantly differ as they use their existing knowledge and bias to fill in the gaps with what they think is likely.

Not OP, but I too am confused. I understood the sketch artist analogy but that didn't seem related to this point:

>They also trained the model with non-blackhole images. Since the output of the model was approximately the same, this indicates that the resulting output picture doesnt look like what we think a black hole looks like just because it was trained with black hole images. It likely really looks like that.

If you are feeding non-blackhole images in and getting blackhole results out, wouldn't that be indicative of an over-trained model? Her other analogy was we can't rule out that there is an elephant at the center of the galaxy, but it sounds like if you feed a picture of an elephant in you'll get a picture of a blackhole out?

This is ensuring that the model is not over trained.

They also showed that when they fed in simulated sparse measurements based on real full images of generic things, they got back fuzzy versions of the real image. [1] So if you put in a sparsely captured elephant (if for instance there was one at the center of the galaxy) you'd get an image of the elephant out, not this black hole.

To complete the artist analogy, imagine that the suspect that is being drawn by each artist is some stereotypical American. The description given to the artists doesnt say that, it just describes how the person looks. One of the three sketch artists is American and the others are Chinese and Ethiopian.

If the American draws a stereotypical American, how can you be sure that the drawing is accurate and thats not just what he assumed the person would look like because everyone he has ever seen looks like that?

You look at what the other two draw. If they both draw the same stereotypical American, even though they have no knowledge of what a stereotypical American looks like, you can be pretty sure that they determined that based on the description provided to them. The actual data.

They did still likely utilize some of their knowledge about what humans in general look like though. This is analogous to how the model uses its training on what a generic image looks like. For instance, maybe several sparse pixels of the same value are likely to have pixels of that same value between them. The model puts things like this together and spits out a picture of what we think a black hole looks like even though its never seen a black hole before.

[1] https://youtu.be/BIvezCVcsYs?t=685

From what I understand, the training input images are just to establish the relationship between sparse data points and full image, regardless of subject matter. Since they were getting the black hole picture out of the trained model regardless of how it was trained, it's likely that the model was producing accurate results of what the "camera" was pointed at. If they had pointed it at an elephant, the model would have produced a picture of something elephant-like because it was somewhat accurately reconstructing a full image from sparse data points.

> getting the black hole picture out of the trained model regardless of how it was trained

Did they try to feed random noise into their trained image builder?

I suspect that the output of that trained image builder is always the same "black hole", even with random noise as an input.

Probably not with random noise. With random noise there is literally no connection between pixels. With any actual picture there are connections. Like for instance a pixel is more likely to be the value of its neighbor or nearly so than any random value. This follows from the fact that the pictures are of actual objects with physical properties that determine the value of the pixel that maps to them. Most of the image can be characterized by continuous gradients with occasional edges.

I think if you trained with random noise you would get random noise output.

They're not just training the model to make pictures from nothing. They're training the model to make pictures from an input.

So I assume they're simulating what an input would look like of, say, a planet or astroid or elephant or whatever, given that it was viewed through the relevant type of sensor system. Then when they feed in the black hole sensor data, they get pictures that look like the black holes we imagined. Even if we never told the model what a black hole looks like.

> that effect is, they trained different versions of the model with different sets of complete images

What does training mean?

I thought that the training means to adjust Neural Network until it learns to convert our input into expected output of "complete image".

But if thaining means to teach the model to produce expected "complete image", then how is it possible that "the output of the model was approximately the same" [for different training "complete image"s]?

Training is to feed in thousands of sets of {sparse sample of actual image, actual image}. The model is adjusted until the total difference between the output image and the actual image is minimized across all training images.

The output images are approximately the same because the model is "looking" at training images at a lower level that we do. The talk says they chop the images up into small pieces. So the model never "sees" the full shapes that are in the full images. It only sees small local features. I guess it turns out that these smaller pieces are pretty generic in that they are common between images of black holes and everything else. The curve of an elephant trunk looks similar to the curve of an event horizon if you cut it out in a small enough piece.

Perhaps if they didnt do this step, then the model would be more sensitive to the images its trained on.

Kathie said [1] "What you can do is to use methods where you [have] do not need any calibration whatsoever and you can still can get pretty good results. So here on the bottom at the top is the truth image, and this is simulated data, as we are increasing the amount of amplitude error and you can see here ... it's hard to see ... but it breaks down once you add too much gain here. But if we use just closure quantities - we are invariant to that. So that really, actually, been a really huge step for the project, because we had such bad gains. " [1] https://youtu.be/UGL_OL3OrCE?t=1177

Hmm I skimmed the paper on the algorithm this morning and didn't get the impression they trained the model on other images. I thought they jointly estimated patches that make up the image and penalized deviations from these patches (i.e. estimated a sparse basis). I haven't watched the Ted talk yet though

They may have changed approaches since this two year old TED talk was made.

If you want to understand that, do not listen to TED talks. They have a terrible format and are designed to make people feel smart rather than impart knowledge.

I had the same issue. It seemed like with many plausible solutions, there is some bias in the image. I agree there are some reasonable constraints, like it should be energetic stuff in a sphere around a dark sphere, but how many solutions would have fit that criteria?