SyntaxHighlighter

Showing posts with label pleiades. Show all posts
Showing posts with label pleiades. Show all posts

Friday, October 13, 2017

Using OpenRefine with Pleiades

This past summer, DC3's Ryan Baumann developed a reconciliation service for Pleiades. He's named it Geocollider. It has two manifestations:

  • Upload a CSV file containing placenames and/or longitude/latitude coordinates, set matching parameters, and get back a CSV file of possible matches.
  • An online Application Programming Interface (API) compatible with the OpenRefine data-cleaning tool.
The first version is relatively self-documenting. This blog post is about using the second version with OpenRefine.

Reconciliation


I.e., matching (collating, aligning) your placenames against places in Pleiades.

Running OpenRefine against Geocollider for reconciliation purposes is as easy as:
When you've worked through the results of your reconciliation process and selected matches, OpenRefine will have added the corresponding Pleiades place URIs to your dataset. That may be all you want or need (for example, if you're preparing to bring your own dataset into the Pelagios network) ... just export the results and go on with your work. 

But if you'd like to actually get information about the Pleiades places, proceed to the next section.

Augmentation


I.e., pulling data from Pleiades into OpenRefine and selectively parsing it for information to add to your dataset.

Pleiades provides an API for retrieving information about each place resource it contains. One of the data formats this API provides is JSON, which is a format with which OpenRefine is designed to work. The following recipe demonstrates how to use the General Refine Expression Language to extract the "Representative Location" associated with each Pleiades place. 

Caveat: this recipe will not, at present, work with the current Mac OSX release of OpenRefine (2.7), even though it should and hopefully eventually will.  It has not been tested with the current releases for Windows and Linux, but they probably suffer from the same limitations as the OSX release. More information, including a non-trivial technical workaround, may be had from OpenRefine Issue 1265. I will update this blog post if and when a resolution is forthcoming.

1. Create a new column containing Pleiades JSON. 

Assuming your dataset is open in an OpenRefine project and that it contains a column that has been reconciled using Geocollider, select the drop-down menu on that column and choose "Edit column" -> "Add column by fetching URLs ..."

Screen capture of OpenRefine column drop-down menu: add column by fetching URLs

In the dialog box, provide a name for the new column you are about to create. In the "expression" box, enter a GREL expression that retrieves the Pleiades URL from the reconciliation match on each cell and appends the string "/json" to it:
cell.recon.match.id + "/json"

Screen capture of OpenRefine dialog box: add column by fetching URLs

OpenRefine retrieves the JSON for each matched place from Pleiades and inserts it into the appropriate cell in the new column. 

2. Create another new column by parsing the representative longitude out of the JSON.

From the drop-down menu on the column containing JSON, select "Edit column" -> "Add column based on this column..."
Screen capture of OpenRefine column drop-down menu: add column based on this column


In the dialog box, provide a name for the new column. In the expression box, enter a GREL expression that extracts the longitude from the reprPoint object in the JSON:
value.parseJson()['reprPoint'][0]

Screen capture of OpenRefine column dialog box: add column based on this column


Note that the reprPoint object contains a two-element list, like:
[ 37.328382, 38.240638 ]
Pleiades follows the GeoJSON specification in using the longitude, latitude ordering of elements in coordinate pairs so, to get the longitude, you use the index (0) for the first element in the list.

3. Create a column for the latitude

Use the method explained in step 2, but select the second list item from reprPoint (index=1).

4. Carry on ...

Your data set in OpenRefine will now look something like this:
screen capture showing portion of an OpenRefine table that includes an ancient toponym, JSON retrieved from Pleiades, and latitude and longitude values extracted from that JSON


Saturday, April 9, 2016

Stable Orbits or Clear Air Turbulence: Capacity, Scale, and Use Cases in Geospatial Antiquity


I delivered the following talk on 8 April 2016 at the Mapping the Past: GIS Approaches to Ancient History conference at the University of North Carolina at Chapel Hill. Update (19 April 2016): video is now available on YouTube, courtesy of the Ancient World Mapping Center.

How many of you are familiar with Jo Guldi's on-line essay on the "Spatial Turn" in western scholarship? I highly recommend it. It was published in 2011 as a framing narrative for the Spatial Humanities website, a publication of the Scholar's Lab at the University of Virginia. The website was intended partly to serve as a record of the NEH-funded Institute for Enabling Geospatial Scholarship. That Institute, organized in a series of three thematic sessions, was hosted by the Scholars Lab in 2009 and 2010. The essay begins as follows:
“Landscape turns” and “spatial turns” are referred to throughout the academic disciplines, often with reference to GIS and the neogeography revolution ... By “turning” we propose a backwards glance at the reasons why travelers from so many disciplines came to be here, fixated upon landscape, together. For the broader questions of landscape – worldview, palimpsest, the commons and community, panopticism and territoriality — are older than GIS, their stories rooted in the foundations of the modern disciplines. These terms have their origin in a historic conversation about land use and agency.
Professor Guldi's essay takes us on a tour through the halls of the Academy, making stops in a variety of departments, including Anthropology, Literature, Sociology, and History. She traces the intellectual innovations and responses -- prompted in no small part by the study and critique of the modern nation state -- that iteratively gave rise to many of the research questions and methods that concern us at this conference. I don't think it would be a stretch to say that not only this conference but its direct antecedents and siblings -- the Ancient World Mapping Center and its projects, the Barrington Atlas and its inheritors -- are all symptoms of the spatial turn.

So what's the point of my talk this evening? Frankly, I want to ask: to what degree do we know what we're doing? I mean, for example, is spatial practice a subfield? Is it a methodology?  It clearly spans chairs in the Academy. But does it answer -- better or uniquely? -- a particular kind of research question? Is spatial inquiry a standard competency in the humanities, or should it remain the domain of specialists? Does it inform or demand a specialized pedagogy? Within ancient studies in particular, have we placed spatially informed scholarship into a stable orbit that we can describe and maintain, or are we still bumping and bouncing around in an unruly atmosphere, trying to decide whether and where to land?

Some will recognize in this framework questions -- or should we say anxieties -- that are also very much alive for the digital humanities. The two domains are not disjoint. Spatial analysis and visualization are core DH activities. The fact that the Scholar's Lab proposed and the NEH Office of Digital Humanities funded the Geospatial Institute I mentioned earlier underscore this point.

So, when it comes to spatial analysis and visualization, what are our primary objects of interest? "Location" has to be listed as number one, right? Location, and relative location, are important because they are variables in almost every equation we could care about. Humans are physical beings, and almost all of our technology and interaction -- even in the digital age -- are both enabled and constrained by physical factors that vary not only in time, but also in three-dimensional space. If we can locate people, places, and things in space -- absolutely or relatively -- then we can open our spatial toolkit. Our opportunities to explore become even richer when we can access the way ancient people located themselves, each other, places, and things in space: the rhetoric and language they used to describe and depict those locations.

The connections between places and between places and other things are also important. The related things can be of any imaginable type: objects, dates, events, people, themes. We can express and investigate these relationships with a variety of spatial and non-spatial information structures: directed graphs and networks for example. There are digital tools and methods at our disposal for working with these mental constructs too, and we'll touch on a couple of examples in a minute. But I'd like the research questions, rather than the methods, to lead the discussion.

When looking at both built and exploited natural landscapes, we are often interested in the functions humans impart to space and place. These observations apply not only to physical environments, but also to their descriptions in literature and their depictions in art and cartography. And so spatial function is also about spatial rhetoric, performance, audience, and reception.

Allow me a brief example: the sanctuary of Artemis Limnatis at Volimnos in the Tayegetos mountains (cf. Koursoumis 2014; Elliott 2004, 74-79 no. 10). Its location is demonstrated today only by scattered architectural, artistic, and epigraphic remains, but epigraphic and literary testimony make it clear that it was just one of several such sanctuaries that operated at various periods and places in the Peloponnese.  Was this ancient place of worship located in a beautiful spot, evocative of the divine? Surely it was! But it -- and its homonymous siblings -- also existed to claim, mark, guard, consecrate, and celebrate political and economic assertions about the land it overlooked. Consequently, the sanctuary was a locus of civic pride for the Messenians and the Spartans, such that -- from legendary times down to at least the reign of Vespasian -- it occasioned both bloodshed and elite competition for the favor of imperial powers. Given the goddess's epithet (she is Artemis Of The Borders), the sanctuary's location, and its history of contentiousness, I don't think we're surprised that a writer like Tacitus should take notice of delegations from both sides arriving in Rome to argue for and against the most recent outcome in the struggle for control of the sanctuary. I can't help but imagine him smirking as he drops it into the text of his Annals (4.43), entirely in indirect discourse and deliberately ambiguous of course about whether the delegations appeared before the emperor or the Senate. It must have given him a grim sort of satisfaction to be able to record a notable interaction between Greece and Rome during the reign of Tiberius that also served as a metaphor for the estrangement of emperor and senate, of new power and old prerogatives.

Epigraphic and literary analysis can give us insight into issues of spatial function, and so can computational methods. The two approaches are complementary, sometimes informing, supporting, and extending each other, other times filling in gaps the other method leaves open. Let's spend some time looking more closely at the computational aspects of spatial scholarship.

A couple of weeks ago, I got to spend some time talking to Lisa Mignone at Brown about her innovative work on the visibility of temples at Rome with respect to the valley of the Tiber and the approaches to the city. Can anyone doubt that, among the factors at play in the ancient siting and subsequent experience of such major structures, there's a visual expression of power and control at work? Mutatis mutandis, you can feel something like it today if you get the chance to walk the Tiber at length. Or, even if you just go out and contemplate the sight lines to the monuments and buildings of McCorkle Place here on the UNC campus. To be sure, in any such analysis there is a major role for the mind of the researcher ... in interpretation, evaluation, narration, and argument, and that researcher will need to be informed as much as possible by the history, archaeology, and literature of the place. But, depending on scale and the alterations that a landscape has undergone over time, there is also the essential place of viewshed analysis. Viewsheds are determined by assessing the visibility of every point in an area from a particular point of interest. Can I see the University arboretum from the north-facing windows of the Ancient World Mapping Center on the 5th floor of Davis Library? Yes, the arboretum is in the Center's viewshed. Well, certain parts of it anyway. Can I see the Pit from there? No. Mercifully, the Pit is not in the Center's viewshed.

In one methodological respect, Professor Mignone's work is not new. Viewshed analysis has been widely used for years in archaeological and historical study, at levels ranging from the house to the public square to the civic territory and beyond. I doubt anyone could enumerate all the published studies without a massive amount of bibliographical work. Perhaps the most well known -- if you'll permit an excursion outside the domain of ancient studies -- is Anne Kelly Knowles' work (with multiple collaborators) on the Battle of Gettysburg. What could the commanders see and when could they see it? There's a fascinating, interactive treatment of the data and its implications published on the website of Smithsonian Magazine.

Off the top of my head, I can point to a couple of other examples in ancient studies. Though their mention will only scratch the surface of the full body of work, I think they are both useful examples. There's Andrew Sherrat's 2004 treatment of Myceneae, which explores the site's visual and topographical advantages in an accessible, online form. It makes use of cartographic illustration and accessible text to make its points about strategically and economically interesting features of the site.

I also recall a poster by James Newhard and several collaborators that was presented at the 2012 meeting of the Archaeological Institute of America. It reported on the use of viewshed analysis and other methods as part of an integrated approach to identifying Byzantine defensive systems in North Central Anatolia. The idea here was that the presence of a certain kind of viewshed -- one offering an advantage for surveillance of strategically useful landscape elements like passes and valleys -- might lend credance to the identification of ambiguous archaeological remains as fortifications. Viewshed analysis is not just revelatory, but can also be used for predictive and taxonomic tasks.

In our very own conference, we'll hear from Morgan Di Rodi and Maria Kopsachelli about their use of viewshed analysis and other techniques to refine understanding of multiple archaeological sites in northwest Greece. So we'll get to see viewsheds in action!

Like most forms of computational spatial analysis, viewshed work is most rigorously and uniformly accomplished with GIS software, supplied with appropriately scaled location and elevation data. To do it reliably by hand for most interesting cases would be impossible. These dependencies on software and data, and the know-how to use them effectively, should draw our attention to some important facts. First of all, assembling the prerequisites of non-trivial spatial analysis is challenging and time consuming. More than once, I've heard Prof. Knowles say that something like ninety percent of the time and effort in a historical GIS project goes into data collection and preparation.  Just as we depend on the labor of librarians, editors, philologists, Renaissance humanists, medieval copyists, and their allies for our ability to leverage the ancient literary tradition for scholarly work, so too we depend on the labor of mathematicians, geographers, programmers, surveyors, and their allies for the data and computational artifice we need to conduct viewshed analysis. This inescapable debt -- or, if you prefer, this vast interdisciplinary investment in our work -- is a topic to which I'd like to return at the end of the talk.

Before we turn our gaze to other methods, I'd like to talk briefly about other kinds of sheds. Watershed analysis -- the business of calculating the entire area drained and supplied by a particular water system -- is a well established method of physical geography and the inspiration for the name viewshed. It has value for cultural, economic, and historical study too, and so should remain on our spatial RADAR. In fact, Melissa Huber's talk on the Roman water infrastructure under Claudius will showcase this very method.

Among Sarah Bond's current research ideas is a "smells" map of ancient Rome. Where in the streets of ancient Rome would you have encountered the odors of a bakery or a latrine a fullonica. And -- God help you -- what would it have smelled like? Will it be possible at some point to integrate airflow and prevailing wind models with urban topography and location data to calculate "smellsheds" or "nosescapes" for particular installations and industries? I sure hope so! Sound sheds ought to be another interesting possibility; we ought to look for leadership to the work of people like Jeff Veitch who is investigating acoustics and architecture at Ostia, and the Virtual Paul's Cross project at North Carolina State.

Every bit as interesting as what the ancients could see, and from where they could see it, is the question of how they saw things in space and how they described them. Our curiosity about ancient geographic mental models and worldview drives us to ask questions like ones Richard Talbert has been asking: did the people living in a Roman province think of themselves as "of the province" in the way modern Americans think of themselves as being North Carolinians or Michiganders? Were the Roman provinces strictly administrative in nature, or did they contribute to personal or corporate identity in some way? Though not a field that has to be plowed only with computers, questions of ancient worldview do sometimes yield to computational approaches.

Consider, for example, the work of Elton Barker and colleagues under the rubric of the Hestia project. Here's how they describe it:
Using a digital text of Herodotus’s Histories, Hestia uses web-mapping technologies such as GIS, Google Earth and the Narrative TimeMap to investigate the cultural geography of the ancient world through the eyes of one of its first witnesses. 
In Hestia, word collocation -- a mainstay of computational text analysis -- is brought together and integrated with location-based measures to interrogate not only the spatial proximity of places mentioned by Herodotus, but also the textual proximities of those place references. With these keys, the Hestia team opens the door to Herodotus' geomind and that of the culture he lived in: what combinations of actual location, historical events, cultural assumptions, and literary agenda shape the mention of places in his narrative?

Hestia is not alone in exploring this particular frontier. Tomorrow we'll hear from Ryan Horne about his collaborative work on the Big Ancient Mediterranean project. Among its pioneering aspects is the incorporation of data about more than the collocation of placenames in primary sources and the relationships of the referenced places with each other. BAM also scrutinizes personal names and the historical persons to whom they refer. Who is mentioned with whom where? What can we learn from exploring the networks of connection that radiate from such intersections?

The introduction of a temporal axis into geospatial calcuation and visualization is also usually necessary and instructive in spatial ancient studies, even if it still proves to be more challenging in standard GIS software than one might like. Amanda Coles has taken on that challenge, and will be telling us more about what it's helped her learn about the interplay between warfare, colonial foundations, road building, and the Roman elites during the Republic.

Viewsheds, worldviews, and temporality, oh my!

How about spatial economies? How close were sources of production to their markets? How close in terms of distance? How close in terms of travel time? How close in terms of cost to move goods?

Maybe we are interested in urban logistics. How quickly could you empty the colosseum? How much bread could you distribute to how many people in a particular amount of time at a particular place? What were the constraints and capacities for transport of the raw materials? What do the answers to such questions reveal about the practicality, ubiquity, purpose, social reach, and costs of communal activities in the public space? How do these conclusions compare with the experiences and critiques voiced in ancient sources?

How long would it take a legion to move from one place to another in a particular landscape? What happens when we compare the effects of landscape on travel time with the built architecture of the limes or the information we can glean about unit deployment patterns from military documents like the Vindolanda tablets or the ostraca from Bu Njem?

The computational methods involved in these sorts of investigations have wonderful names, and like the others we've discussed, require spatial algorithms realized in specialized software. Consider cost surfaces: for a particular unit of area on the ground, what is the cost in time or effort to pass through it? Consider network cost models: for specific paths between nodal points, what is the cost of transit? Consider least cost path analysis: given a cost surface or network model, what is the cheapest path available between two points?

Many classicists will have used Orbis: The Stanford Geospatial Network Model of the Roman World. The Orbis team, assembled by Walter Scheidel, has produced an online environment in which one can query a network model of travel costs between key nodal points in the Roman world, varying such parameters as time of year and mode of transport. This model, and its digital modes of access, bring us to another vantage point. How close were two places in the Roman world, not as the crow flies, not in terms of miles along the road, but as the boat sailed or the feet walked.

Barbora Weissova is going to talk to us tomorrow about her work in and around Nicaea. Among her results, she will discuss another application of Least Cost Path Analysis: predicting the most likely route for a lost ancient roadway.

It's not just about travel, transport, and cost. Distribution patterns are of interest too, often combined with ceramic analysis, or various forms of isotopic or metallurgical testing, to assess the origin, dissemination, and implications of ancient objects found in the landscape. Inscriptions, coins, portable antiquities, architectural and artistic styles, pottery, all have been used in such studies. Corey Ellithorpe is going to give us a taste of this approach in numismatics by unpacking the relationship between Roman imperial ideology and regional distribution patterns of coins.

I'd like to pause here for just a moment and express my hope that you'll agree with the following assessment. I think we are in for an intellectual feast tomorrow. I think we should congratulate the organizers of the conference for such an interesting, and representative, array of papers and presentations. That there is on offer such a tempting smorgasbord is also, of course, all to the credit of the presenters and their collaborators. And surely it must be a factor as we consider the ubiquity and disciplinarity of spatial applications in ancient studies.

Assiduous students of the conference program will notice that I have neglected yet to mention a couple of the papers. Fear not, for they feature in the next section of my talk, which is -- to borrow a phrase from Meghan Trainor and Kevin Kadish -- all about that data.

So, conference presenters, would you agree with the dictum I've attributed to Anne Knowles? Does data collection and preparation take up a huge chunk of your time?

Spatial data, particularly spatial data for ancient studies, doesn't normally grow on trees, come in a jar, or sit on a shelf. The ingredients have to be gathered and cleaned, combined and cooked. And then you have to take care of it, transport it, keep track of it, and refashion it to fit your software and your questions. Sometimes you have to start over, hunt down additional ingredients, or try a new recipe. This sort of iterative work -- the cyclic remaking of the experimental apparatus and materials -- is absolutely fundamental to spatially informed research in ancient studies.

If you were hoping I'd grind an axe somewhere in this talk, you're in luck. It's axe grinding time.

There is absolutely no question in my mind that the collection and curation of data is part and parcel of research. It is a research activity. It has research outcomes. You can't answer questions without it. If you aren't surfacing your work on data curation in your CV, or if you're discounting someone else's work on data curation in decisions about hiring, tenure, and promotion, then I've got an old Bob Dylan song I'd like to play for you.

  • Archive and publish your datasets. 
  • Treat them as publications in your CV. 
  • Write a review of someone else's published dataset and try to get it published. 
  • Document your data curation process in articles and conference presentations.

Right. Axes down.

So, where does our data come from? Sometimes we can get some of it in prepared form, even if subsequent selection and reformatting is required. For some areas and scales, modern topography and elevation can be had in various raster and vector formats. Some specialized datasets exist that can be used as a springboard for some tasks. It's here that the Pleiades project, which I direct, seeks to contribute. By digitizing not the maps from the Barrington Atlas, but the places and placenames referenced on those maps and in the map-by-map directory, we created a digital dataset with potential for wide reuse. By wrapping it in a software framework that facilitates display, basic cartographic visualization, and collaborative updates, we broke out of the constraints of scale and cartographic economy imposed by the paper atlas format. Pleiades now knows many more places than the Barrington did, most of these outside the cultures with which the Atlas was concerned. More precise coordinates are coming in too, as are more placename variants and bibliography. All of this data is built for reuse. You can collect it piece by piece from the web interface or download it in a number of formats. You can even write programs to talk directly to Pleiades for you, requesting and receiving data in a computationally actionable form. The AWMC has data for reuse too, including historical coastlines and rivers and map base materials. It's all downloadable in GIS-friendly formats.

But Pleiades and the AWMC only help for some things. It's telling that only a couple of the projects represented at this conference made use of Pleiades data. That's not because Pleiades is bad or because the authors didn't know about Pleiades or the Center. It's because the questions they're asking require data that Pleiades is not designed to provide.

It's proof of the point I'd like to underline: usually -- because your research question is unique in some way, otherwise you wouldn't be pursuing it -- you're going to have to get your hands dirty with data collection.

But before we get dirty, I'm obliged to point out that, although Pleiades has received significant, periodic support from the National Endowment for the Humanities since 2006, the views, findings, conclusions, or recommendations expressed in this lecture do not necessarily reflect those of the National Endowment for the Humanities.

We've already touched on the presence of spatial language in literature. For some studies, the placenames, placeful descriptions, and narratives of space found in both primary and secondary sources constitute raw data we'd like to use. Identifying and extracting such data is usually a non-trivial task, and may involve a combination of manual and computational techniques, the latter depending on the size and tractability of the textual corpus in question and drawing on established methods in natural language processing and named entity recognition. It's here we may encounter "geoparsing" as a term of art. Many digital textual projects and collections are doing geoparsing: individual epigraphic and papyrological publications using the Text Encoding Initiative and EpiDoc Guidelines; the Perseus Digital Library; the Pelagios Commons by way of its Recogito platform. The China Historical GIS is built up entirely from textual sources, tracking each placename and each assertion of administrative hierarchy back to its testimony.

For your project, you may be able to find geoparsed digital texts that serve your needs, or you may need to do the work yourself. Either way, some transformation on the results of geoparsing is likely to be necessary to make them useful in the context of your research question and associated apparatus.

Relevant here is Micah Myers's conference paper. He is going to bring together for us the analysis and visualization of travel as narrated in literature. I gather from his abstract that he'll show us not only a case study of the process, but discuss the inner workings of the on-line publication that has been developed to disseminate the work.

Geophysical and archaeological survey may supply your needs. Perhaps you'll have to do fieldwork yourself, or perhaps you can collect the information you need from prior publications or get access to archival records and excavation databases. Maybe you'll get lucky and find a dataset that's been published into OpenContext, the British Archaeology Data Service, or tDAR: the Digital Archaeological Record. But using this data requires constant vigilance, especially when it was collected for some purpose other than you own. What were the sampling criteria? What sorts of material were intentionally ignored? What circumstances attended collection and post-processing?

Sometimes the location data we need comes not from a single survey or excavation, but from a large number of possibly heterogeneous sources. This will be the case for many spatial studies that involve small finds, inscriptions, coins, and the like. Fortunately, many of the major documentary, numismatic, and archaeological databases are working toward the inclusion of uniform geographic information in their database records. This development, which exploits the unique identifying numbers that Pleiades bestows on each ancient place, was first championed by Leif Isaksen, Elton Barker, and Rainer Simon of the Pelagios Commons project. If you get data from a project like the Heidelberg Epigraphic Databank, papyri.info, the Arachne database of the German Archaeological Institute, the Online Coins of the Roman Empire, or the Perseus Digital Library, you can count on being able to join it easily with Pleiades data and that of other Pelagios partners. Hopefully this will save some of us some time in days to come.

Sometimes what's important from a prior survey will come to us primarily through maps and plans. Historical maps may also carry information we'd like to extract and interpret. There's a whole raft of techniques associated with the scanning, georegistration, and georectification (or warping) of maps so that they can be layered and subjected to feature tracing (or extraction) in GIS software. Some historic cartofacts -- one thinks of the Peutinger map and medieval mappae mundi as examples -- are so out of step with our expectations of cartesian uniformity that these techniques don't work. Recourse in such cases may be had to first digitizing features of interest in the cartesian plane of the image itself, assigning spatial locations to features later on the basis of other data. Digitizing and vectorizing plans and maps resulting from multiple excavations in order to build up a comprehensive archaeological map of a region or site also necessitates not only the use of GIS software but the application of careful data management practices for handling and preserving a collection of digital files that can quickly grow huge.

We'll get insight into just such an effort tomorrow when Tim Shea reports on Duke's "Digital Athens Project".

Let's not forget remote sensing! In RS we use sensors -- devices that gather information in various sections of the electro-magnetic spectrum or that detect change in local physical phenomena. We mount these sensors on platforms that let us take whatever point of view is necessary to achieve the resolution, scale, and scope of interest: satellites, airplanes, drones, balloons, wagons, sleds, boats, human hands. The sensors capture emitted and reflected light in the visible, infrared, and ultraviolet wavelengths or magnetic or electrical fields. They emit and detect the return of laser light, radio frequency energy, microwaves, millimeter waves, and, especially underwater, sound waves. Specialized software is used to analyze and convert such data for various purposes, often into rasterized intensity or distance values that can be visualized by assigning brightness and color scales to the values in the raster grid. Multiple images are often mosaicked together to form continuous images of a landscape or 360 degree seamless panoramas.

Remotely sensed data facilitate the detection and interpretation of landforms, vegetation patterns, and physical change over time, revealing or enhancing understanding of built structures and exploited landscapes, as well as their conservation. This is the sort of work that Sarah Parcak has been popularizing, but it too has decades of practice behind it. In 1990, Tom Sever's dissertation reported on a remote-sensing analysis of the Anasazi road system, revealing a component of the built landscape that was not only invisible on the ground, but that demonstrates that the Anasazi were far more willing than even the Romans to create arrow-straight roads in defiance of topographical impediments. More recently, Prof. Sever and his NASA colleague Daniel Irwin have been using RS data for parts of Guatemala, Honduras, and Mexico, to distinguish between vegetation that thrives in alkaline soils and vegetation that doesn't. Because of the Mayan penchant for coating monumental structures with significant quantities of lime plaster, this data has proved remarkably effective in the locating of previously unknown structures beneath forest canopy. The results seem likely to overturn prevailing estimates of the extent of Mayan urbanism, illustrating a landscape far more cleared and built upon than heretofore proposed (cf. Sever 2003).

Given the passion with which I've already spoken about the care and feeding of data, you'll be unsurprised to learn that I'm really looking forward to Nevio Danelon's presentation tomorrow on the capture and curation of remotely sensed data in a digital workflow management system designed to support visualization processes.

I think it's worth noting that both Professor Parcak's recently collaborative work on a possible Viking settlement in Newfoundland, as well as Prof. Sever's dissertation, represent a certain standard in the application of remote sensing to archaeology. RS analysis is tried or adopted for most archaeological survey and excavation undertaken today. The choice of sensors, platforms, and analytical methods will of course vary in response to landscape conditions, expected archaeological remains, and the realities of budget, time, and know-how.

Similarly common, I think, in archaeological projects is the consideration of geophysical, alluvial, and climatic features and changes in the study area. The data supporting such considerations will come from the kinds of sources we've already discussed, and will have to be managed in appropriate ways. But it's in this area -- ancient climate and landscape change -- that I think ancient studies has a major deficit in both procedure and data. Computational, predictive modeling of ancient climate, landscape, and ground cover has made no more than tentative and patchy inroads on the way we think about and map the ancient historical landscape. That's a deficit that needs addressing in an interdisciplinary and more comprehensive way.

I'd be remiss if, before moving on to conclusions, I kept the focus so narrowly on research questions and methods that we miss the opportunity to talk about pedagogy, public engagement, outreach, and cultural heritage preservation. Spatial practice in the humanities is increasingly deeply involved in such areas. The Ancient World Mapping Center's Antiquity a-la Carte website enables users to create and refine custom maps from Pleiades and other data that can then be cited, downloaded, and reused. It facilitates the creation of map tests, student projects, and maps to accompany conference presentations and paper submissions.

Meanwhile, governments, NGOs, and academics alike are brining the full spectrum of spatial methods to bear as they try to prevent damage to cultural heritage sites through assessment, awareness, and intervention. The American Schools of Oriental Research conducts damage assessments and site monitoring with funding in part from the US State Department. The U.S. Committee of the Blue Shield works with academics to prepare geospatial datasets that are offered to the Department of Defense to enhance compliance with the 1954 Hague Convention for the Protection of Cultural Property in the Event of Armed Conflict.

These are critical undertakings as well, and should be considered an integral part of our spatial antiquity practice.

So, how should we gather up the threads of this discussion so we can move on to the more substantive parts of the conference?

I'd like to conclude as I began, by recommending an essay. In this case, I'm thinking of Bethany Nowiskie's recent essay on "capacity and care" in the digital humanities. Bethany is the former director of UVA's Scholars Lab. She now serves as Director of the Digital Library Federation at the Council on Library and Information Resources. I had the great good fortune to hear Bethany deliver a version of this essay as a keynote talk at a project director's meeting hosted by the NEH Office of Digital Humanities in Washington in September of last year. You can find the essay version on her personal website.

Bethany thinks the Humanities must expand its capacity in order not only to survive the 21st century, but to contribute usefully to its grand challenges. To cope with increasing amounts and needs for data of every kind. To move gracefully in analysis and regard from large scales to small ones and to connect analysis at both levels. To address audiences and serve students in an expanding array of modes. To collaborate across disciplines and heal the structurally weakening divisions that exist between faculty and "alternate academics", even as the entire edifice of faculty promotion and tenure threatens to shatter around us.

What is Bethany's prescription? An ethic of care. She defines an ethic of care as "a set of practices", borrowing the following quotation from the political scientist Joan Tronto:
a species of [collective] activity that includes everything we do to maintain, contain, and repair our world, so that we can live in it as well as possible.
I think our practice of spatial humanities in ancient studies is just such a collective activity. We don't have to turn around much to know that we are cradled in the arms and buoyed up on the shoulders of a vast cohort, stretching back through time and out across the globe. Creating data and handing it on. Debugging and optimizing algorithms. Critiquing ideas and sharpening analytical tools.

The vast majority of projects on the conference schedule, or that I could think of to mention in my talk, are explicitly and immediately collaborative.

And we can look around this room and see like-minded colleagues galore. Mentors. Helpers. Friends. Comforters. Makers. Guardians.

And we have been building the community infrastructure we need to carry on caring about each other and about the work we do to explain the human past to the human present and to preserve that understanding for the human future. We have centers and conferences and special interest groups and training sessions. We involve undergraduates in research and work with interested people from outside the academy. We have increasingly useful datasets and increasingly interconnected information systems. Will all these things persist? No, but we get to play a big role in deciding what and when and why.

So if there's a stable orbit to be found, I think it's in continuing to work together and to do so mindfully, acknowledging our debts to each other and repaying them in kind.

I'm reminded of a conversation I had with Scott Madry, back in the early aughts when we were just getting the Mapping Center rolling and Pleiades was just an idea. As many of you know, Scott together with Carole Crumley and numerous other collaborators here at UNC and beyond, have been running a multidimensional research project in Burgundy since the 1970s. At one time or another the Burgundy Historical Landscapes project has conducted most of the kinds of studies I've mentioned tonight, all the while husbanding a vast and growing store of spatial and other data across a daunting array of systems and formats.

I think that the conversation I'm remembering with Scott took place after he'd spent a couple of hours teaching undergraduate students in my seminar on Roman roads and land travel how to do orthophoto analysis the old fashioned way: with stereo prints and stereoscopes. He was having them do the Sarah Parcak thing: looking for crop marks and other indications of potentially buried physical culture. After the students had gone, Scott and I were commiserating about the challenges of maintaining and funding long-running research projects. I was sympathetic, but know now that I really didn't understand those challenges then. Scott did, and I remember what he said. He said: "We were standing on that hill in Burgundy twenty years ago, and as we looked around I said to Carol: 'somehow, we are going to figure out what happened here, no matter how long it takes.'"

That's what I'm talking about.

Monday, September 28, 2015

Updated: Lightning talk on Pleiades at NEH

UPDATED 4 March 2016: Followers of the ISAW News Blog will have seen the recent piece on the award of an NEH grant for new work on the Pleiades gazetteer and the follow-up piece about my intended participation at a meeting of similarly funded project directors. NEH's Office of Digital Humanities intends to has posted video of all the talks online on YouTube. You can drop into my 3 minutes, starting at 16:01:10, but meantime Or, if you'd prefer to read rather than listen, here are words I read and the slides I showed during the 3 minutes allotted to my "lightning talk". 

Slides are at Slideshare.

[SLIDE 1]
I am here today as a representative of the Pleiades community, an international group of volunteers who build and maintain the most comprehensive geospatial dataset for antiquity available today. Like many people in this room, we believe that the study of the past is a fundamental aspect of the Humanities endeavor. It's essential to understanding what it means to be human today, and to envisioning how we might be better humans in the future. Our collective past is geographically entangled: "where" is the stage on which the human drama is played, and it's an important analytical variable in every field of the past-oriented Humanities: history, archaeology, linguistics, text analysis, and so on. We also believe that the places and spaces known to and inhabited by our ancestors are the precious and fragile property of every person alive today, no matter whether we can still see and touch that heritage, or only just imagine it. 
[SLIDE 2]
So, scholars, students, and the public need free and open data about their ancient geography. They need it in order to learn about the past, to advance research, and to inform conservation. They also need it to connect digital images, texts, and other information across the web, regardless of where that information is created or hosted. Unsurprisingly, these same individuals have the collective skill and energy necessary to create and improve geographic information, if only we can put good tools in their hands.
That's what Pleiades does. It combines web-enabled public participation with peer review and editorial oversight in order to identify and describe ancient places and spaces. It continuously enables and draws upon the work of individuals, groups, and their computational agents as a core component of a growing, public scholarly communications network. 
[SLIDE 3]
And now, thanks to the Endowment, its reviewers, and the National Humanities Council, we have the opportunity to supercharge that network. Responding to recent adjustments in the guidelines for the digital humanities implementation grants, we requested funds to retool Pleiades. A decade of growth and diversification has left our web application underpowered and unreliable even as more users and external projects look to Pleiades as a source of information and a venue for publication. We need more power to address the most urgent needs articulated by the community: accelerated content creation and review, faster dissemination and discovery, display support for phones and tablets, expanded spatial and temporal coverage, flexible modeling of spatial relationships, comprehensive and customized access and preservation. 
Thanks to NEH, we can continue going where no daughters of Atlas have gone before!

Wednesday, August 21, 2013

Get a List of Machine Tags from the Flickr API

I want to know what Pleiades machine tags are in use on photos throughout Flickr (more background here). I thought I'd learn how to ask for that information from the Flickr API via a script. I requested and got an API key (see http://www.flickr.com/help/api/). I set up a Python virtual environment and git repository for the project. I went looking for Python code that already implemented interaction with the API and settled (somewhat arbitrarily) on Beej's Python Flickr API kit (now maintained by Sybren Stüvel). Then used pip install flickrapi to get the package.

Here's a command-line session running the script and showing its output:

(pleiades-flickr)darkstar:pleiades-flickr paregorios$ python src/listptags.py 
pleiades:atteststo is used on 15 photos in Flickr
pleiades:denotes is used on 1 photos in Flickr
pleiades:depcits is used on 2 photos in Flickr
pleiades:depicts is used on 7229 photos in Flickr
pleiades:findspot is used on 2197 photos in Flickr
pleiades:finspot is used on 2 photos in Flickr
pleiades:foundat is used on 1 photos in Flickr
pleiades:observedat is used on 3 photos in Flickr
pleiades:origin is used on 225 photos in Flickr
pleiades:place is used on 970 photos in Flickr
pleiades:places is used on 19 photos in Flickr
pleiades:where is used on 119 photos in Flickr
(pleiades-flickr)darkstar:pleiades-flickr paregorios$ 


Here's the code (version at github):

#!/usr/bin/env python
"""
A Flickr tag bot
"""

import argparse
import flickrapi
import json
import logging as l
import os
import sys
import traceback

from myflickr import API_KEY, NAMESPACE_DEFAULT

SCRIPT_DESC = "poll machine tags from flickr"

def main ():
    """ Unleash the bot! """

    global args
    global l

    flickr = flickrapi.FlickrAPI(API_KEY)
    resp = flickr.machinetags_getPairs(namespace=args.namespace, format="json")
    if resp[:14] == "jsonFlickrApi(":
        jstr = resp[14:-1]
        j = json.loads(jstr)
        ptags = [(p['_content'], p['usage']) for p in j['pairs']['pair']]
        for ptag in ptags:
            print "%s is used on %s photos in Flickr" % ptag


if __name__ == "__main__":
    try:
        parser = argparse.ArgumentParser(description=SCRIPT_DESC, formatter_class=argparse.ArgumentDefaultsHelpFormatter)
        parser.add_argument ("-n", "--namespace", default=NAMESPACE_DEFAULT, help="namespace to use in requesting machine tags")
        parser.add_argument ("-v", "--verbose", action="store_true", default=False, help="verbose output")
        args = parser.parse_args()
        if args.verbose:
            l.basicConfig(level=l.DEBUG)
        else:
            l.basicConfig(level=l.WARNING)
        main()
        sys.exit(0)
    except KeyboardInterrupt, e: # Ctrl-C
        raise e
    except SystemExit, e: # sys.exit()
        raise e
    except Exception, e:
        print "ERROR, UNEXPECTED EXCEPTION"
        print str(e)
        traceback.print_exc()
        os._exit(1)


Comments, questions, and constructive criticism welcomed!

Thursday, April 18, 2013

Citing Sources in Digital Annotations

I'm collaborating with other folks both in and outside ISAW on a variety of digital scholarly projects in which Linked Open Data is playing a big role. We're using the Resource Description Framework (RDF) to provide descriptive information for, and make cross-project assertions about, a variety of entities of interest and the data associated with them (places, people, themes/subjects, creative works, bibliographic items, and manuscripts and other text-bearing objects). So, for example, I can produce the following assertions in RDF (using the Terse RDF Triple Language, or TuRTLe):

<http://syriaca.org/place/45> a <http://geovocab.org/spatial#Feature> ;
  rdfs:label "Serugh" ;
  rdfs:comment "An ancient city where Jacob of Serugh was bishop."@en ;
  foaf:primaryTopicOf <http://en.wikipedia.org/wiki/Suruç> ;
  owl:sameAs <http://pleiades.stoa.org/places/658405#this> .

This means: 'There's a resource identified with the Uniform Resource Identifier (URI) "http://syriaca.org/place/45" about which the following is asserted:
(Folks familiar with what Sean Gillies has done for the Pleiades RDF will recognize my debt to him in the what proceeds.)

But there are plenty of cases in which just issuing a couple of triples to encode an assertion about something isn't sufficient; we need to be able to assign responsibility/origin for those assertions and to link them to supporting argument and evidence (i.e., standard scholarly citation practice). For this purpose, we're very pleased by the Open Annotation Collaboration, whose Open Annotation Data Model was recently updated and expanded in the form of a W3C Community Draft (8 February 2013) (the participants in Pelagios use basic OAC annotations to assert geographic relationships between their data and Pleiades places).


A basic OADM annotation uses a series of RDF triples to link together a "target" (the thing you want to make an assertion about) and a "body" (the content of your assertion). You can think of them as footnotes. The "target" is the range of text after which you put your footnote number (only in OADM you can add a footnote to any real, conceptual, or digital thing you can identify) and the "body" is the content of the footnote itself. The OADM draft formally explains this structure in section 2.1. This lets me add an annotation to the resource from our example above (the ancient city of Serugh) by using the URI "http://syriaca.org/place/45" as the target of an annotation) thus:
<http://syriaca.org/place/45/anno/desc6> a oa:Annotation ;
  oa:hasBody <http://syriaca.org/place/45/anno/desc6/body> ;
  oa:hasTarget <http://syriaca.org/place/45> ;
  oa:motivatedBy oa:describing ;
  oa:annotatedBy <http://syriaca.org/editors.xml#tcarlson> ;
  oa:annotatedAt "2013-04-03T00:00:01Z" ;
  oa:serializedBy <https://github.com/paregorios/srpdemo1/blob/master/xsl/place2ttl.xsl> ;
  oa:serializedAt "2013-04-17T13:35:05.771-05:00" .

<http://syriaca.org/place/45/anno/desc6/body> a cnt:ContentAsText, dctypes:Text ;
  cnt:chars "an ancient town, formerly located near Sarug."@en ;
  dc:format "text/plain" ;

I hope you'll forgive me for not spelling that all out in plain text, as all the syntax and terms are explained in the OADM. What I'm concerned about in this blog post is really what the OADM doesn't explicitly tell me how to do, namely: show that the annotation body is actually a quotation from a published book. The verb oa:annotatedBy lets me indicate that the annotation itself was made (i.e., the footnote was written) by a resource identified by the URI "http://syriaca.org/editors.xml#tcarlson". If I'd given you a few more triples, you could have figured out that that resource is a real person named Thomas Carlson, who is one of the editors working on the Syriac Reference Portal project. But how do I indicate (as he very much wants to do because he's a responsible scholar and has no interest in plagiarizing anyone) that he's deliberately quoting a book called The Scattered Pearls: A History of Syriac Literature and Sciences? Here's what I came up with (using terms from Citation Typing Ontology and the DCMI Metadata Terms):
<http://syriaca.org/place/45/anno/desc7/body> a cnt:ContentAsText, dctypes:Text ;
  cnt:chars "a small town in the Mudar territory, between Ḥarran and Jarabulus. [Modern name, Suruç (tr.)]"@en ;
  dc:format "text/plain" ;
  cito:citesAsSourceDocument <http://www.worldcat.org/oclc/255043315> ;
  dcterms:biblographicCitation  "The Scattered Pearls: A History of Syriac Literature and Sciences, p. 558"@en .

The addition of the triple containing cito:citesAsSourceDocument lets me make a machine-actionable link to the additional structured bibliographic data about the book that's available at Worldcat (but it doesn't say anything about page numbers!). The addition of the triple containing dcterms:bibliographicCitation lets me provide a human-readable citation.

I'd love to have feedback on this approach from folks in the OAC, CITO, DCTERMS, and general linked data communities. Could I do better? Should I do something differently?


The SRP team is currently evaluating a sample batch of such annotations, which you're also welcome to view. The RDF can be found here. These files are generated from the TEI XML here using the XSLT here.

Thursday, December 6, 2012

Pleiades Hackday: Improving Descriptions

A Pleiades hack day is now underway in the Pleiades IRC channel. We've decided to focus on cleaning up descriptions for Pleiades place resources on the island of Sardinia. The following query gives you most of the Pleiades content for Sardinia and Corsica:

http://tinyurl.com/aotxaum

Here's our process:


  1. Pick a place to improve
  2. Drop the title and URL into the IRC channel, so everyone knows you're working on it
  3. Check out a working copy of the place
  4. In your working copy, write a better description and connect the place to the place resource for the island you're working on.
  5. Submit for review
  6. Ping paregorios or servilius_ahala in IRC to get it reviewed and published

We're also building an FAQ on the subject of "What Makes a Good Pleiades Description?".

Thursday, October 4, 2012

Pleiades Machine Tags for Blog Posts? Yes!

So, a few minutes ago I noticed a new post in my feed reader from a blog I've admired for a while: Javier Andreu Pintado's Oppida Imperii Romani. I've thought for a long time that I ought to get in touch with him (we don't know each other from Adam as far as I know) and see if we could figure out a more-or-less automated way to get his posts to show up on the associated Pleiades pages.

Then it hit me:

Why can't we just use labels incorporating Pleiades IDs like we've been doing with machine tags on Flickr and query the Blogger API to get the associated posts?

Why not indeed. It turns out it just works.

To test, I added the string "pleiades:depicts=579885" as a label on my blog post from last December, "Pleiades, Flickr, and the Ancient World Image Bank" (Since that tag is used in an example in that post. I recognize that the blog post doesn't actually depict that place, which is what that label term ought to mean, but this is just a test).

Then I went to the Google APIs Explorer page for the Blogger "list posts" function (which I found by googling) and entered by blog's ID and the label string in the appropriate fields.



And, in a matter of milliseconds, I got back a JSON representation of my blog post.



So now I'm thinking we might explore the possibility of creating a widget on Pleiades place pages to feature blog posts tagged like this from selected blogs. It appears that, to execute the API queries against Blogger, we have to do them blog-by-blog with known IDs, but that's probably OK anyway so we can curate the process of blog selection and prevent spam.

It occurs to me that the Pelagios community might be interested in looking at this approach in order to build a gateway service to inject blog posts into the Pelagios network.

And while I'm name-checking, I wonder if any Wordpress aficionados out there can come up with a functionally equivalent mechanism.

Monday, February 6, 2012

Playing with PELAGIOS: The GAWD is Live

The is the lastest in an on-going series chronicling my dalliances with data published by the PELAGIOS project partners.

I think it's safe to say that, thanks to the PELAGIOS partner institutions, that we do have a Graph of Ancient World Data (GAWD) on the web. It's still in early stages, and one has to do some downloading, unzipping, and so forth to engage with it at the moment, but indeed the long-awaited day has dawned.

Here's the perspective, as of last Friday, from the vantage point of Pleiades. I've used SPARQL to query the GAWD for all information resources that the partners claim (via their RDF data dumps) are related to Pleiades information resources. I.e., I'm pulling out a list of information resources about texts, pictures, objects, grouped by their relationships to what Pleiades knows about ancient places (findspot, original location, etc.). I've sorted that view of the graph by the titles Pleiades gives to its place-related information resources and generated an HTML view of the result. It's here for your browsing pleasure.

Next Steps and Desiderata

For various technical reasons, I'm not yet touching the data of a couple of PELAGIOS partners (CLAROS and SPQR), but the will hopefully be resolved soon. I still need to dig into figuring out what Open Context is doing on this front. Other key resources -- especially those emanating from ISAW -- are not yet ready to produce RDF (but we're working on it).

There are a few things I'd like the PELAGIOS partners to consider/discuss adding to their data:

  • Titles/labels for the information resources (using rdfs:label?). This would make it possible for me to produce more intuitive/helpful labels for users of my HTML index. Descriptions would be cool too. As would some indication of the type of thing(s) a given resource addresses (e.g., place, statue, inscription, text)
  • Categorization of the relationships between their information resources and Pleaides information resources. Perhaps some variation of the terms originally explored by Concordia (whence the GAWD moniker), as someone on the PELAGIOS list has already suggested.
What would you like to see added to the GAWD? What would you do with it?

Thursday, February 2, 2012

Playing with PELAGIOS: Dealing with a bazillion RDF files

Latest in a Playing with PELAGIOS series

Some of the PELAGIOS partners distribute their annotation RDF in a relatively small number of files. Others (like SPQR and ANS) have a very large number of files. This makes the technique I used earlier for adding triples to the database ungainly. Fortunately, 4store provides some command line methods for loading triples.

First, stop the 4store http server (why?):
$ killall 4s-httpd
Try to import all the RDF files.  Rats!
$ 4s-import -a pelagios *.rdf
-bash: /Applications/4store.app/Contents/MacOS/bin/4s-import: Argument list too long
Bash to the rescue (but note that doing one file at a time has a cost on the 4store side):
$ for f in *.rdf; do 4s-import -av pelagios $f; done
Reading <file:///Users/paregorios/Documents/files/P/pelagios-data/coins/0000.999.00000.rdf>
Pass 1, processed 10 triples (10)
Pass 2, processed 10 triples, 8912 triples/s
Updating index
Index update took 0.000890 seconds
Imported 10 triples, average 4266 triples/s
Reading <file:///Users/paregorios/Documents/files/P/pelagios-data/coins/0000.999.101.rdf>
Pass 1, processed 11 triples (11)
Pass 2, processed 11 triples, 9856 triples/s
Updating index
Index update took 0.000936 seconds
Imported 11 triples, average 4493 triples/s
Reading <file:///Users/paregorios/Documents/files/P/pelagios-data/coins/0000.999.10176.rdf>
Pass 1, processed 8 triples (8)
Pass 2, processed 8 triples, 6600 triples/s
Updating index
Index update took 0.000892 seconds
Imported 8 triples, average 3256 triples/s
... 
This took a while. There are 86,200 files in the ANS annotation batch.

Note the use of the -a option on 4s-import to ensure the triples are added to the current contents of the database, rather than replacing them! Note also the -v option, which is what gives you the report (otherwise, it's silent and that makes my ctrl-c finger twitchy).

Now, back to the SPARQL mines.

Wednesday, February 1, 2012

Playing with PELAGIOS: Arachne was easy after nomisma

Querying Pleiades annotations out of Arachne RDF was as simple as loading the Arachne Objects by Places RDF file into 4store the same way I did nomisma and running the same SPARQL query.  Cost: 5 minutes. Now I know about 29 objects in the Arachne database that they think are related to Akragas/Agrigentum. For example:

Playing with PELAGIOS: Nomisma

So, I want to see how hard it is to query the RDF that PELAGIOS partners are putting together. The first experiment is documented below.

Step 1: Set up a Triplestore (something to load the RDF into and support queries)

Context: I'm a triplestore n00b. 

I found Jeni Tennison's Getting Started with RDF and SPARQL Using 4store and RDF.rb and, though I had no interest in messing around with Ruby as part of this exercise, the recommendation of 4store as a triplestore sounded good, so I went hunting for a Mac binary and downloaded it.

Step 2: Grab RDF describing content in Nomisma.org

Context: I'm a point-and-click expert.

I downloaded the PELAGIOS-conformant RDF data published by Nomisma.org at http://nomisma.org/nomisma.org.pelagios.rdf.

Background: "Nomisma.org is a collaborative effort to provide stable digital representations of numismatic concepts and entities, for example the generic idea of a coin hoard or an actual hoard as documented in the print publication An Inventory of Greek Coin Hoards (IGCH)."

Step 3: Fire up 4store and load in the nomisma.org 

Context: I'm a 4store n00b, but I can cut and paste, read and reason, and experiment.

Double-clicked the 4store icon in my Applications folder. It opened a terminal window.

To create and start up an empty database for my triples, I followed the 4store instructions and Tennison's post (mutatis mutandis) and so typed the following in the terminal window ("pelagios" is the name I gave to my database; you could call yours "ray" or "jay" if you like):
$ 4s-backend-setup pelagios
$ 4s-backend pelagios
Then I started up 4store's SPARQL http server and aimed it at the still-empty "pelagios" database so I could load my data and try my hand at some queries:
$ 4s-httpd pelagios
Loading the nomisma data was then as simple as moving to the directory where I'd saved the RDF file and typing:
$ curl -T nomisma.org.pelagios.rdf 'http://localhost:8080/data/http://nomisma.org/nomisma.org.pelagios.rdf/'
Note how the URI base for nomisma items is appended to the URL string passed via curl. This is how you specify the "model URI" for the graph of triples that gets created from the RDF.

Step 4: Try to construct a query and dig out some data.

Context: I'm a SPARQL n00b, but I'd done some SQL back in the day and XML and namespaces are pretty much burned into my soul at this point. 

Following Tennison's example, I pointed my browser at http://localhost:8080/test/. I got 4store's SPARQL test query interface. I googled around looking grumpily at different SPARQL "how-tos" and "getting starteds" and trying stuff and pondering repeated failure until this worked:

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX oac: <http://www.openannotation.org/ns/>

SELECT ?x
WHERE {
 ?x oac:hasBody <http://pleiades.stoa.org/places/462086> .
} 

That's "find the ID of every OAC Annotation in the triplestore that's linked to Pleiades Place 462086" (i.e., Akragas/Agrigentum, modern Agrigento in Sicily). It's a list like this:
  • http://nomisma.org/nomisma.org.pelagios.rdf#igch1910-agrigentum-5
  • http://nomisma.org/nomisma.org.pelagios.rdf#igch2089-agrigentum-24
  • http://nomisma.org/nomisma.org.pelagios.rdf#igch2101-agrigentum-32
  • ...
51 IDs in all.

But what I really want is a list of the IDs of the nomisma entities themselves so I can go look up the details and learn things. Back to the SPARQL mines until I produced this:
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX oac: <http://www.openannotation.org/ns/>

SELECT ?nomismaid
WHERE {
 ?x oac:hasBody <http://pleiades.stoa.org/places/462086> .
 ?x oac:hasTarget ?nomismaid .
} 

Now I have a list of 51 nomisma IDs: one for the mint and 50 coin hoards that illustrate the economic network in which the ancient city participated (e.g., http://nomisma.org/id/igch2081).

Cost: about 2 hours of time, 1 cup of coffee, and three favors from Sebastian Heath on IRC.

Up next: Arachne, the object database of the Deutsches Archäologisches Institut.



Tuesday, December 20, 2011

Pleiades, Flickr, and the Ancient World Image Bank

Many of you are already aware that Pleiades, Flickr, and the Ancient World Image Bank have joined forces to link together online, open-access imagery and ancient geographical information. This blog post is intended to answer some lingering questions that users and potential contributors have been asking about the process.

Other Blog Posts
How to Construct a Pleiades Machine Tag

In Flickr, you add the machine tag the same way you add regular tags when editing an individual image or a set or group in the organizer. The machine tag should use the following syntax:
pleiades:TERM=#####
where "TERM" is one of the recognized terms (originally from the Concordia Thesaurus, aka the Graph of Ancient World Data, or GAWD, terms) listed below and "#####" is the numeric identifier of the Pleiades place you wish to associate with the photo.

You can get the identifier by visiting pleiades.stoa.org then searching for and finding the place. Copy the numeric portion of the URL of the place page and paste it into your tag.

So, for example, if I wanted to tag a photo that "depicts" ancient Athens (or a portion thereof), I'd visit Pleiades and search for Athens. I'd find this place page: http://pleiades.stoa.org/places/579885/. So, I'd grab 579885 and construct the following machine tag for use in Flickr:
pleiades:depicts=579885
Recognized Terms in Pleiades Machine Tags

TERMS for use in Pleiades machine tags should begin with a lowercase letter. The following TERMS are recognized in Pleiades machine tags:
depicts
the photo so tagged can be said to "depict" the referenced ancient place or a significant or exemplary portion thereof
(this term is equivalent to CIDOC CRM p62 "depicts")
findspot
the photo so tagged shows an object that was first found in modern times at the referenced ancient place
(this term is particularly useful for items now in museums or elsewhere, especially those no longer at the initial place of finding)
origin
the photo so tagged shows an object that is believed, with reasonably high certainty, to have been originally located or produced at the referenced ancient place
(this can differ from the findspot, as when an inscription or other object was moved in antiquity)
observedAt
the photo so tagged shows an object that was observed in modern times at the referenced ancient place
(the implication being that the place observed is neither the modern findspot nor the presumed original location; I suspect this term will rarely need to be used to link photos with Pleiades place resources)
where
the photo so tagged is related in some way to the referenced ancient place, but for some unspecified reason no more specific relationship can be asserted
(this term should not be used unless none of above terms are deemed to be appropriate)
place
this term is DEPRECATED; it was originally used in exploring the Pleiades machine tag idea (and is highlighted in my previous blog post). Its semantics are assumed to be equivalent to "where". If at all possible, photos carrying this tag should be updated to use one of the more specific terms above.

Please note that pleiades:places=##### (i.e., with a plural) is not a recognized machine tag. Its behavior in Flickr or Pleiades is undefined. So is the behavior of any Pleiades machine tag with a misspelling or a term not included in the list above. ("finspot" seems to be a popular typo at present).

Any photo tagged with the proper Pleiades machine tag syntax and one of the terms above will be noticed by Pleiades and picked up in the summary counts and links on individual place resource pages. In order for a photo to be considered for the Pleiades Places group on Flickr (and therefore as a "portrait image" for a Pleiades place), the photo must be tagged with a pleiades:depicts tag.

How do I Add Pleiades Machine Tags Quickly?

It would be unreasonable of us to ask a prolific photographer with an extensive, well-tagged collection already on Flickr to go through an individually add appropriate machine tags by hand. Fortunately, Flickr provides a mechanism for easy batch editing of tags.
  1. Suppose that you have tagged a large number of your photos with the name of the ancient site (e.g., Halicarnassus)
  2. Visit the following link: http://www.flickr.com/photos/me/alltags/; it will give you an alphabetical list of all your tags.
  3. Find the name of the site and click on the corresponding link. You'll see all the photos thus tagged in your photostream.
  4. Find and click on the "Change this tag" link (Really. Skip "Edit these in a batch" link for now).
  5. Insert the cursor after your existing tag string, type a space, then type or paste in the desired Pleiades machine tag (read the fine print on that page for an extended explanation of what's going on).
  6. Click the "save" button. Flickr will go off and add the new tag to all those images at once.
  7. If you'd rather be more selective about which photos you want to add a tag to, you can choose the "Edit these in a batch" link I told you to skip above, then paste the Pleiades machine tag into the tag lists associated with only those images you wish to update.
Why Can't We Just Use Geotagging That's Already in the Photos?

Some photographers have geotagged their photos, either using a GPS-enabled digital camera or some method of post-processing. I was recently asked on twitter why we're putting people to all the trouble above when we could just use the geotagging? There are several reasons:
  • Not everybody's photos are machine-tagged.
  • Many ancient sites are coincident with urban areas (or areas of natural beauty or places where someone took a picture of their dog) and so mere proximity to an ancient site can't be interpreted as indicating a given photo is relevant to a nearby Pleiades place.
  • Horizontal precision and accuracy can vary widely in geotagged photos as a function of the geotagging method used and the interests and skill of the person doing the geotagging. As a result, a photo might be geotagged at a location closer to another, unrelated pleiades place.
  • The horizontal precision and accuracy of Pleiades coordinates also varies widely given the varying sources from which it derives and the subsequent coordinate extraction methods. This makes the process of proximity correlation even more fraught.
This is not to say that we're not interested in exploring uses for geotagged photos in Flickr (or supporting a geotag-your-photo-using-Pleiades-coordinates tool), but I hope this discussion helps explain why we like the machine-tag approach for indicating relevance.

Please let me know, via comments here, if you have additional questions or suggestions.