BaedekAR
大綱
Introduction
In this section, you’ll be building an AR guidebook — a Baedeker — for paintings and photos in galleries, museums, or any place that displays 2D art.
Importing the Reference Images
The first step in building BaedekAR is providing it with the set of images that it should detect and recognize. You’ll do that in this step.
Detecting Reference Images in the Real World
With the reference images imported into the app, it’s time to take the first step and detect those images in the real world, making note of their location and size.
func startARSession() {
// Make sure that we have an AR resource group.
guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else {
fatalError("This app doesn't have an AR resource group named 'AR Resources'!")
}
// Set up the AR configuration.
config.worldAlignment = .gravityAndHeading
config.detectionImages = referenceImages
// Start the AR session.
sceneView.session.run(config, options: [.resetTracking, .removeExistingAnchors])
statusViewController.scheduleMessage("Look around for art!",
inSeconds: 7.5,
messageType: .contentPlacement)
} // This delegate method gets called whenever the node corresponding to
// a new AR anchor is added to the scene.
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
// We only want to deal with image anchors, which encapsulate
// the position, orientation, and size, of a detected image that matches
// one of our reference images.
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
let imageName = referenceImage.name ?? "[Unknown]"
let isArtImage = !imageName.hasPrefix("block_this-")
var statusMessage = ""
if isArtImage {
statusMessage = "Found \(artworkDisplayNames[imageName] ?? "artwork")."
} else {
statusMessage = "Unpleasant image blocked."
}
DispatchQueue.main.async {
self.statusViewController.cancelAllScheduledMessages()
self.statusViewController.showMessage(statusMessage)
}
// Draw the appropriate plane over the image.
updateQueue.async {
}
DispatchQueue.main.async {
self.statusViewController.cancelAllScheduledMessages()
self.statusViewController.showMessage(statusMessage)
}
}Making Detected Images Tappable
Now that the app can detect the reference images in the real world, let’s make it so that the user can tap them to find out more about them.
// Draw the appropriate plane over the image.
updateQueue.async {
var planeNode = SCNNode()
if isArtImage {
// If the detected artwork is one that we’d like to highlight (and one which we’d
// like the user to tap to find out more), draw an “artwork” plane and
// the name of the artwork over the image.
planeNode = self.createArtworkPlaneNode(withReferenceImage: referenceImage, andImageName: imageName)
let nameNode = self.createArtworkNameNode(withImageName: imageName)
node.addChildNode(nameNode)
} else {
// If the detected artwork is one that we’d like to obscure,
// draw a “blocker” plane over the image.
planeNode = self.createBlockerPlaneNode(withReferenceImage: referenceImage, andImageName: imageName)
planeNode.name = self.blockedName
}// Create a translucent plane that will overlay the detected artwork,
// which the user can tap for more information.
// The plane will flash momentarily when it first appears.
func createArtworkPlaneNode(withReferenceImage referenceImage: ARReferenceImage,
andImageName imageName: String) -> SCNNode {
// "Flash" the plane so the user is aware that it’s now available.
// Called when the plane first appears.
let flashPlaneAction = SCNAction.sequence([
.wait(duration: 0.25),
.fadeOpacity(to: 0.85, duration: 0.25),
.fadeOpacity(to: 0.25, duration: 0.25),
.fadeOpacity(to: 0.85, duration: 0.25),
.fadeOpacity(to: 0.25, duration: 0.25),
.fadeOpacity(to: 0.85, duration: 0.25),
.fadeOpacity(to: 0.25, duration: 0.25),
])
// Draw the plane.
let plane = SCNPlane(width: referenceImage.physicalSize.width * 1.5,
height: referenceImage.physicalSize.height * 1.5)
let planeNode = SCNNode(geometry: plane)
planeNode.opacity = 0.25
planeNode.runAction(flashPlaneAction)
planeNode.name = imageName
return planeNode
} @objc func handleScreenTap(sender: UITapGestureRecognizer) {
let tappedSceneView = sender.view as! ARSCNView
let tapLocation = sender.location(in: tappedSceneView)
let tapsOnPlane = tappedSceneView.hitTest(tapLocation,
options: nil)
if !tapsOnPlane.isEmpty {
let firstObject = tapsOnPlane.first
if let firstObjectName = firstObject?.node.name {
performSegue(withIdentifier: "showArtNotes",
sender: firstObjectName)
}
}
}Annotating Detected Images With AR text
Let’s improve the app by displaying the name of the detected image using AR text. We’ll also make use of billboard constraints to make sure that the text is always facing the user, so that it’s easy to read.
// Create a text node to display the name of an artwork.
func createArtworkNameNode(withImageName imageName: String) -> SCNNode {
let textScaleFactor: Float = 0.15
let textFont = "AvenirNext-BoldItalic"
let textSize: CGFloat = 0.2
let textDepth: CGFloat = 0.02
let artworkNameText = SCNText(string: artworkDisplayNames[imageName],
extrusionDepth: 0.02)
artworkNameText.font = UIFont(name: textFont, size: textSize)?.withTraits(traits: .traitBold)
artworkNameText.alignmentMode = kCAAlignmentCenter
artworkNameText.firstMaterial?.diffuse.contents = UIColor.orange
artworkNameText.firstMaterial?.specular.contents = UIColor.white
artworkNameText.firstMaterial?.isDoubleSided = true
artworkNameText.chamferRadius = CGFloat(textDepth)
let artworkNameTextNode = SCNNode(geometry: artworkNameText)
artworkNameTextNode.scale = SCNVector3(textScaleFactor, textScaleFactor, textScaleFactor)
artworkNameTextNode.name = imageName
let (minBound, maxBound) = artworkNameText.boundingBox
artworkNameTextNode.pivot = SCNMatrix4MakeTranslation((maxBound.x - minBound.x) / 2,
minBound.y,
0)
let billboardConstraint = SCNBillboardConstraint()
billboardConstraint.freeAxes = SCNBillboardAxis.Y
artworkNameTextNode.constraints = [billboardConstraint]
return artworkNameTextNode
}Blocking Images
What if there were images that we wanted to block from the user?
An AR Cure for Clown Phobia
Let’s protect the user from scary clown images by blocking them with a soothing picture of Ray.
// Create an opaque plane featuring the soothing image of Ray Wenderlich,
// which will be used to obscure unwanted images.
// Yes, we’re in “Black Mirror” territory now.
func createBlockerPlaneNode(withReferenceImage referenceImage: ARReferenceImage,
andImageName imageName: String) -> SCNNode {
// Draw the plane.
let plane = SCNPlane(width: referenceImage.physicalSize.width * 2.0,
height: referenceImage.physicalSize.height * 2.0)
let planeNode = SCNNode(geometry: plane)
planeNode.geometry?.firstMaterial?.diffuse.contents = UIImage(named: "ray")
planeNode.name = imageName
return planeNode
}Conclusion
Last updated
Was this helpful?