CINNIC Figures and Tables:

Figure 1. This is an example of a contour created by Make Snake (Braun, 1999). As can be seen, there appears to be a complete circle. However, the circle is created by unconnected Gabor wavelet elements. The mind connects these elements in a phenomenon known as contour integration.

Figure 2. (A) An image is taken (1) and is split into 12 orientation-filtered images (2), which are sent to their own layers in the corresponding group where Each of the 12 preferred orientations are rotated at 15 degrees (3). After interaction the output is collected at a top-level saliency map (4). (B) Interaction between layers is governed by collinearity. More collinear elements excite each other (a and b angles are small) while less collinear elements suppress each other (Alpha and Beta are large). (C) Elements like (1) enhance, elements like (2) suppress, and highly parallel elements can enhance, like in (3).

Figure 3. An important element of the model is a fast plasticity term. In our model we follow the notion of priming via dopamine. (1) A neuron and its neighbor receive input. (2) The neuron on the right sends a signal to the neuron on the left. (3) The left neuron is now primed via dopamine. (4) When the neuron on the left receives another input, it is more likely to cross its firing threshold. This allows contour neurons to propagate activity to other contour neurons that are not directly connected.

Figure 4. This is a conceptual illustration of two suppression groups. Gain in the network is controlled by a Basket GABAergic interneuron like connection scheme. This works by spatially grouping local neurons into groups that are all suppressed by a local interneuron for that group. This creates

Figure 5. (A) A Kernel is generated that dictates the base strength of the connections between neurons in the network. Each kernel slice shown represents the interaction between two neurons given their preferred orientations. Red represents inhibition while green represent excitation. (B) If two neurons are parallel in preference but not collinear, then they inhibit each other. (C) Parallel bars excite if they are close to collinear in preference. The three kernels shown (the same as highlighted in (A)) show the interaction if elements are related to each other as shown by the bars. For instance, if two elements are totally co-linear they would use the first kernel. The next kernel would be used if one element is offset by 15 degrees. (D) This is a side view of the 0 degree offset kernel. The kernel has modest 2^nd and 3^rd order polynomial curvature, which can be observed on close inspection.

Figure 6. This graph illustrates the way in which neurons interact with neurons in other hypercolumns. By mapping the hyper-kernel K over the neuron Alpha,i,j we can find the base synaptic current generated that should be sent to another neuron at the relative position Beta,k,l.

Figure 7. CINNIC works in several phases. The first is to take in a real world image. A Gaussian filter is applied that creates 12 orientation selective images. The image is then rescaled using an image pyramid into 3 different scales. The 12 orientation selective images are then pseudo-convolved and the corresponding region is run with dopamine-like fast plasticity and group suppression over several iterations. The three different scales are then brought back together using a weighted average and combined into a contour saliency map.

Figure 8. The top three images show the results of pseudo-convolution at each of the three scales used. The bottom right image shows the weighted average of the three images. The circles represent what the program feels are the five most salient contour locations. The bottom left image is the input image with the most salient points shown with the red circle on the most salient point and the blue circle on the least salient of the top five.

Figure 9. The program makes a decision as to which of two images has the target in it. The model estimates this decision by taking the probability of a decision as the Poisson of the output at the target. The error is the error function (EFC) of the two distributions for both target and non-target. Target amplitude is changed until error rate is 25% (75% correct). This response marks the relative enhancement.

Figure 10. The algorithm was optimized against observer AM. The pre-optimized output has a similar shape, but approaches the performance results from observer AM following optimization of CINNIC using hill climbing. The decision process from the program yields results that are within 2 standard errors (0.05) at its greatest difference found at a separation of 2 Lambda.

Figure 11. As a collinear element draws closer, its receptive field begins to overlap another element’s region of surround inhibition (red). Here the stimulus element sizes may be compared with the kernel at the 64x64 pixel scale, which are 2.396 Lambda (3 pixels), 4 Lambda (5 pixels) and 5.597 Lambda (7 pixels). The separations for elements shown are at 2.4 Lambda, 1.6 Lambda and .8 Lambda. Here we interacted two single elements with a kernel. As elements get larger and closer, it can be seen that enhancement dips. Careful analysis shows that this is due to overlap of elements into inhibition zones, in the surround, as they move closer. Thus, no special kernel, or neural structure is necessary to create inverse enhancement at very close distances between two elements. This explains the dip in enhancement at close distances observed in CINNIC and by Polat and Sagi.

Figure 12. Input images created by Make Snake are run through CINNIC. The output saliency map is processed to find the five most salient points. These five points are compared with a mask that represents the position of foreground contour elements. This allows the ground truth for such images to be determined with greater ability since foreground elements are controlled.

Figure 13. The results from processing 2000 images from Make Snake by CINNIC are shown. The sum of all images where the most salient point was on a foreground contour is shown in dark gray for each of the Lambda separation conditions. In the experiment all images where the second most salient point was on a foreground element but the first was not are labeled 2^nd and are in a lighter shade of gray. In each condition, the general saliency result can by seen by summing the number of images where a foreground element is among the five most salient points found. At separations between 2.4 Lambda and 3.2 Lambda foreground and background element separation is about the same. At 5.14 Lambda, elements fall beyond the reach of enhancement defined by the finest resolution kernel. Thus, we expect to begin to see a drop off here. There is a slight pick up in enhancement between 3.2 Lambda and 5.14 Lambda perhaps due to optimal separation where elements do not overlap each other’s inhibition regions.

Figure 14. The declining performance of CINNIC at increasing l separation is easy to understand by inspecting the contour images at 1.5, 3.5 and 6.0 Lambda of foreground separation. Casual observation shows that saliency decreases with larger separation of contour elements. At 6.0 l contour elements are almost invisible.

Figure 15. The size of kernels at each of the three scales is shown compared with Make Snake image. The line on the Make Snake image shows the width of each kernel for close reference against an image with foreground separation of 4.5 Lambda which is the same separation as the peak observed in figure 13. As can be seen, when the image is reduced to 16x16, the kernel stretches across much of the image, but with little specificity of affect on the image due to the scale reduction.

Figure 16. The five images shown above demonstrate CINNIC’s sensitivity to junctions in the elemental shapes seen. Here, the most salient point is always on a junction (red circle) and there is always another point of very high saliency (in the top five) on a junction. When not falling on a junction, the most salient point is near the center between two junctions, which is quite possibly the second most important part to find salient. Some of the anomalies observed such as a saliency point in blank space are due to the algorithm blanking out the saliency map as it selects points to prevent it from picking the same point more than once.

Figure 17. These graphs show enhancement of pixels from an image when convolved with an orthogonal slice from the CINNIC kernel. As can be seen, in the top left graph, the corners on L junctions, both one and two pixels wide, are enhanced more than their neighbor pixels and other pixels along the L out to a distance of 4.8 Degrees. Additionally, in the top right, we can see that the corners on bars are enhanced over pixels outside of their receptive field (> 4.8 Degrees) along the same bar as the two parallel edges are separated and additionally as group suppression is added. The bottom row shows that end stops with a point are not enhanced at base group suppression, but as suppression is added, the end point overtakes its three closest neighbors (.8,1.6,2.4 Degrees) when group suppression reaches 150%. This effect is not seen for the non-pointed bar. Thus, the current version of CINNIC is only conditionally sensitive to end-stops. Note, each pixel corresponds to a width of .8 degrees with the 64x64 scale kernel.

Figure 18. The five most salient points are shown in 12 real world images processed by CINNIC (red is most salient, next is orange etc.). Notice the prevalence of representation by facial features, junctions and end-stops.

Figure 19.  Temporal grouping can be explained by fast-plasticity mechanisms. If alternation is strongly correlated then plastic connections are strong and less ambiguous. Also, by the second alternation, all connections are primed unlike uncorrelated alternation where only some connections are primed. As such, correlated temporal alternation would facilitate neurons more strongly than a less correlated temporal structure if it used fast-plasticity based priming.

Table 1. As l separation increases between foreground elements, saliency decreases. For the smaller Gabor sized image, around 75% of all images with a foreground separation of 1.5 to 5 l have a foreground element as one of the top five most salient.  The probability of obtaining such a result at random is far less than .005 percent. For images with larger Gabor elements, almost all the images contain a foreground element that is highly salient. Again the probability is very low suggesting that the null hypothesis should be rejected.

Table 2. Post Hoc analysis of CINNIC for its sensitivity to certain kinds of features again suggests that it is not only sensitive to contours, but junctions as well. This can be seen as the most salient point in 42% of random real world images analyzed lies on a contour junction. Prior probability is not supplied since it is not known by us what the real incidence of contour junctions is in real world images. Thus, the true posterior significance is unknown.