Like most vascular systems in the body, the retinal vasculature develops to metabolically sustain cells. By providing nutrients to the inner part of the retina, the retinal vasculature helps to maintain cell viability that allows humans to see.
1,2 As health issues arise, the shape of the vasculature may change, which can inhibit the delivery of nutrients to different regions of the eye. Fortunately, a quantitative approach for measuring shape has the opportunity to provide significant advances.
3 One study with respect to vasculature, has quantified the increase in retinal blood flow within patients with diabetes mellitus and found a significant increase in blood flow within the retina, though flow is not measured in typical retinal fundus images. These changes in blood flow preceded any rupturing of blood vessels or significant biomarkers. This indicates that there is the potential for vasculature to change due to disease before rupturing.
4 One vascular related disease is diabetic retinopathy (DR). Doctors perform regular screenings to catch the disease before it permanently affects vision, which could lead to blindness if it goes untreated. Standard screening programs capture retinal fundus images, and ophthalmologist experts visually inspect images for leaking blood vessels, microaneurysms, retinal swelling, hemorrhages, cotton wool spots, exudates, retinal ischemia, or neovascularization before diagnosing a patient with DR.
5 Several of these features are shown in
Figure 1. Developing a better way to describe other subtle changes in vasculature could help with training for identification of important vascular cues of DR and allow for diseases to be detected
before irreversible damage occurs to the retina.
As machine learning techniques have become more powerful, more researchers have attempted to use these tools to automate disease diagnosis in the retina.
6,7 With the large number of images that have been recorded to train these algorithms, correlations between the images and other health factors have been explored. One study attempted to predict the systolic blood pressure from the fundus images and used the machine learning tool saliency maps
8 that highlight pixels of the image that contributed most to the prediction. Although 98% of doctors agreed that these maps highlighted the blood vessels, the qualities of the blood vessels that conveyed the information was not known.
9 This further supports the concept that vascular changes occur, but current methods could be significantly improved with language to observe and describe these differences.
Isolating vasculature from the other features within the retinal fundus image forces viewers to focus on the vasculature and consider the features within it. Through generating realistic but synthetic images of retinal vasculature, this work seeks to provide more data to train ophthalmologists and machine learning algorithms to process retinas with a purely vascular approach. Although separating the components of tissue where symptoms could exist may seem counterproductive, it is precisely this separation that may allow a fresh look at an old problem. Currently, machine learning methods are trained on the entire fundus, which can include vascular abnormalities and retinal abnormalities. The causal relationship between these abnormalities is not always clear, and with certain diseases, removing retinal information may hurt diagnosis. With other diseases, this simplification may prove helpful. Isolating retinal vasculature could not only lead to disease diagnosis at a subclinical level but also be developed into a new training model for clinicians.
Generating medical images can also be useful for training doctors, validating image analysis techniques, and producing massive amounts of data needed for machine learning methods. Simulating the generation of blood vessels has been used for surgical training with simulation incorporated so doctors can more accurately experience what would happen when taking certain actions.
10 Additionally, with the increasing demand for large amounts of data from machine learning methods that are not always available, synthetically produced data can be created to help train networks to diagnose patients for various diseases.
11 Although Costa et al.
11 used adversarial neural networks to produce realistic and new retinal fundus images from noise, their approach does not explicitly track how features vary across each of the images, which is important in determining disease diagnosis and training. These features are all implicitly learned by the neural networks.
Generating retinal images using shape grammars, which are described in the next paragraph, offers the potential to create new images that can be used for medical training, where important nuances in image visualization can alter the diagnosis via control over what types of images that are generated. One way this can be accomplished is through adding noise to rules from a shape grammar of existing images through parametric variation. This could produce unique images of a similar style as the original.
Shape grammars are a field originally introduced in architecture by Stiny and Gips,
12 where shapes are identified and then modified on the basis of the application of rules, through successive iteration, to change and build up an overall shape design.
13,14 A shape grammar consists of a set of shapes S and a set of rules R, where the rules can act on the shapes S to generate new shapes S′, where S′ is also known as a language:
\begin{eqnarray}
r:{\rm s}\rightarrow {\rm s^{\prime}},
\nonumber\end{eqnarray}
where
\begin{eqnarray}
r \in R, s \in S, s^{\prime} \in S^{\prime}.
\nonumber\end{eqnarray}
The set of rules can continue to be applied to the shapes, until there are no more valid rules or a termination rule is applied. Shapes can also be parametric, resulting in a succinct grammar representing a potentially infinite shape space, such as the one presented in this article.
Such shape grammars have been shown to be able to capture particular styles, such as villas in the style of Palladio
15 and the prairie houses of Frank Lloyd Wright.
16 In these examples, the shape grammars were used to recreate existing and generate new designs that are of the same style as captured in the grammar language. Cagan and colleagues introduced shape grammars to the engineering design community, and they demonstrated the ability of shape grammars to capture and generate products representative of different brands, including coffee makers,
17 motorcycle designs like Harley-Davidson brands,
18 or cars in the style of Buick,
19 among others.
The goals of applying shape grammars to biologic design are similar to the goals of grammars in architecture and mechanical design. Biologic systems contain patterns and features with some commonalities and some discrepancies. All blood vessel networks deliver nutrients to tissues within the body, but the specific paths of blood vessels in each human are unique. Retinal vasculature also contains common features as well, such as blood vessels coming from the optic nerve and distributing nutrients, but the patterns and shapes are unique for each person and change with time.
As with shape grammars for coffee makers and many other products, which leverage parameters to model functional, as well as varied form, designs, a retinal vascular grammar must also capture common functional qualities, like branching and curvature, but also enable varied generation of the shape of the vascular system. Furthermore, most design capture with shape grammar begins with an initial shape from which the overall design is built, such as the fireplace for the Frank Lloyd Wright prairie houses or the wheelbase of motorcycles. So, too, must a retinal vascular grammar begin with the optic nerve from which all vessels flow.
This article explores how a shape grammar is applied to retinal vasculature by considering only a three-rule grammar. Describing the vasculature and generating new vasculature show the beginning of the possibilities of applying shape grammars for medical imagery. Retinal diseases like diabetic retinopathy and retinopathy of prematurity cause symptoms in the retina due to retinal vascular changes.
2 Identifying changes in the retinal vasculature before retinal disease is clinically apparent affords an avenue of diagnostic capability that could disrupt clinical ophthalmology. They offer an alternative approach more explainable than existing machine learning diagnostic techniques, which implicitly learn only patterns from their training data.
20
The goals of this article are to create a framework based on shape grammars to better understand blood vessel networks and explain patterns that are present. This is done through deriving a simple three-rule grammar and using it to break down existing vascular networks and generate new networks through reapplying these rules with noise. Shape grammars offer the potential to accurately describe features of the vasculature and allow comparison of these features between patients. With image processing techniques, the shape grammar approach could be used to identify issues within the vasculature. These findings may be useful for the detection of ocular disease caused by vasculature and the effects of primary retinal disease on vasculature.