It’s not easy to say why, exactly, but some places are more magical than others. Maybe it’s the sense of history or intriguing culture, or perhaps a complex landscape or even the light in the air. Whatever it is, Taos has it. A five-hour drive from Denver, this town of 5,766 residents somehow feels distinctly foreign: an unpredictable blend of Spanish ancestry, Native American culture, Southwestern artsiness, and dramatic scenery. My wife, Chris, and I have been visiting this high-desert enclave for two decades, but Taos still feels exotic and even a little mysterious. With the mix of languages heard in town, with the way busy streets narrow into lonely dirt roads bordered by high adobe walls, and with its alluring history, this New Mexican jewel surprises us every time we visit.

On our latest trip, we arrange to check in early at Taos’ Palacio de Marquesa, an inn that reopened in summer 2014 after extensive renovations. Most of the guest rooms now boast a sleek aesthetic of nickel and white furnishings with New Mexican accents visible in the dark ceiling beams and colorful woven throws—from nearby Chimayo—on the beds. We chose the inn partly because it welcomed our small dog, Tucker (albeit at a stiff price—$50 per night), and partly because it is nestled into a quiet neighborhood with secluded gardens in the back but is still only a 10-minute walk from downtown.

We drop our stuff in our room and head into town for lunch, cutting through the historic Taos Plaza. The city was settled in the early 1600s—more than two centuries before Denver—when Northern New Mexico was first colonized by Spain, and the main plaza was built for defensive purposes, with all the windows and doors facing inward. Today, those windows are filled with tchotchkes, paintings, and silver jewelry that might attract tourists like us. We wander over to John Dunn Way, a pedestrian alley lined with shops, where the sound and steam of a sizzling wok lure us to the Great Noodles street cart. Behind a colorful chalkboard of specials and a line of spicy condiments, chef Marshall Thompson has been whipping up memorable stir-fry for the past six years. We sit in the sun slurping noodles with barbecue pork and pondering the incongruity of outstanding Asian street food in a town best known for chiles and sopaipillas.

I usually have limited bandwidth for shopping, but Taos’ retail scene is compact and varied—an ideal combination that allows me to enjoy an hour or two of browsing. After lunch, Chris, who collects Día de los Muertos objects, heads straight to Coyote Moon to browse the shelves of arte popular. Next, while she watches Tucker, I check in at independent bookshop Op.Cit Taos, down the street, for its shelves of new, used, and rare tomes. We cross Bent Street to poke around Dwellings Revisited—which feels a little like the flea market you always wished you could find but never do—then wander over to Ledoux Street, a narrow, adobe-walled lane of art galleries and museums that, come December, will be bedecked with luminaria candles. Doubling back through the plaza, we stop at Maison Faurie Antiquités, where Chris chats with the friendly French proprietor. Upon learning Chris once taught high school biology, the owner guides her to display cases of antique surgical instruments and glass eyeballs. He piques her interest, but Chris can’t quite imagine the bizarre collection in our home. Strolling down Paseo del Pueblo, Taos’ main street, we make one more stop: the Alhambra de Taos, a bazaarlike store filled with antiquities and home furnishings sourced from all around the world. More than once we’ve left Alhambra with our car piled high as well, but this time we make it back to the inn with only a few small bags.

Much like the town itself, Taos’ dining scene is full of flavor. We try a new (or new to us) restaurant in the area every time we visit. In Arroyo Seco, a quiet village 15 minutes north of Taos (it’s on the way to Taos Ski Valley, New Mexico’s biggest ski area), we settle into a table at Aceq, named for the “acequia,” or communal ditches, that irrigate the area. The menu is small but diverse; the decor is simple but pleasing; and the food is upscale but approachable. My seared tuna wrapped in a sesame crust is a deftly executed blend of crunchy exterior and tender, warm interior. Chris’ locally raised lamb burger is too dry to rise to the same level, leaving her wishing she’d ordered the house specialty: fried chicken. Since we’re planning a picnic for the next day, we order a box to go. Then we stroll past Arroyo Seco’s 1830s adobe church and peek into potters’ studios as alpenglow lights up Wheeler Peak, the state’s 13,159-foot high point, easily discernible just east of town.

A knock at our guest room door the next morning portends good things: The Palacio de Marquesa has a common area for breakfast but also will deliver eggs, fruit, muffins, juice, coffee, and tea to your room if you so desire. We sip the last of our coffee while sitting at a tiny table outside in the garden and discussing the plan for day two. We consider hiking in the Wild Rivers Recreation Area, where the Rio Grande and Red River meet, or driving the High Road, a fabulous daytrip to the outskirts of Santa Fe that passes numerous artists’ studios and the mystical Santuario de Chimayo church, but rock climbing wins the day. Heading for the Rio Grande Gorge, we drive south and stop for a peek at the San Francisco de Asis mission church, a 200-year-old, fortresslike adobe building in Ranchos de Taos that’s been a favorite subject of many artists, including Georgia O’Keeffe. At the hamlet of Pilar, we turn down a small road and enter the two-year-old Rio Grande del Norte National Monument, which spans more than 50 miles of river (starting at the Colorado border). Most tourists see the river only from the Rio Grande Gorge Bridge across the nearly 600-foot-deep canyon, or perhaps while white-water rafting near Pilar. But mountain bikers, hikers, kayakers, fishermen, and rock climbers will find much to discover in the sprawling new monument.

We hike about a mile out the West Rim Trail to a line of basalt cliffs that climbers call Utopian Vistas. The gorge below us is so deep and straight it looks like a cosmic sword has slashed the volcanic plains. The autumn sun is surprisingly hot, and when we take a break from climbing, Aceq’s fried chicken—delicious as promised—makes our fingers so greasy we can no longer hang on to the black rock. We decide to head back to town for a look into Taos’ colorful past.

Taos is much smaller and a little rougher around the edges than Santa Fe, the state capital, 90 minutes to the south. Outside its busy center, this is still a rural community, and we love exploring the older neighborhoods, where million-dollar homes coexist with ramshackle mini-ranches. So instead of driving straight to the Millicent Rogers Museum, our afternoon destination, we set our GPS and wind along narrow back roads, twisting around horse pastures bordered by rows of tightly lashed sapling poles known as coyote fences. The museum, a sprawling former home resting in a field of sagebrush, houses the collection of Millicent Rogers, an oil heiress who moved to Taos in the 1940s after a failed romance with Clark Gable. Each room here is dedicated to a different Southwestern craft: jewelry, blankets, the 600-year-old baskets and decorated pottery of Ancestral Puebloans, carved santos figures and wooden religious panels called “retablos,” and beautiful wooden crosses with straw inlays—an inexpensive way to create golden iconography on the Spanish frontier. If you want a one-stop introduction to Northern New Mexico crafts, there’s no better place.

It’s almost dinnertime, and though Chris and I love to make new discoveries during our travels, there’s one stop in Taos we insist on making every time: the Love Apple. Located about a mile from Taos Plaza, this 13-table restaurant lives inside a wood-ceilinged, whitewashed, 19th-century chapel with a crooked cupola. We warm our hands around an outdoor fire pit as we wait to be seated. The Love Apple serves a local and seasonal menu of Northern New Mexico cuisine—beef-bone broth posole, lamb albondigas, chicken confit tacos—but I only need to glance at the menu to verify my favorite dish is still there: rainbow trout slathered with lime butter, wrapped in corn husks, grilled, and drizzled with chipotle crème. The word “perfecto” comes to mind.

On day three we only have a few hours before starting our drive home, and once again, we find our list of Taos to-do’s is too long. We’d love to hang out longer to tour the Harwood Museum of Art, eat at Lambert’s (a high-end favorite for steak), or perhaps enjoy a happy hour margarita on the patio of the pink adobe Taos Inn. But we decide to leave such modern delights for our next trip and instead pay a visit to the oldest community in the area: Taos Pueblo, a UNESCO World Heritage Site at the foot of the mountains just outside town, where Tiwa-speaking Native Americans have lived continuously for as many as 1,000 years. The jumbled, five-story North House and South House complexes have no plumbing or electricity, and only about 150 people live here full time, but many others still maintain their families’ traditional rooms. A guide leads us on a half-hour tour of the tribe’s Catholic church, the sacred Rio Pueblo running through the small enclave, a ceremonial outdoor “horno” (a domed adobe oven), and the ruins of an older church destroyed during the Taos Revolt in 1847, three years before New Mexico became a U.S. territory. Our guide describes her childhood days running around the pueblo, but she gently refuses to answer questions about the rituals that occur inside those dark pueblo rooms. Some Taos secrets aren’t meant to be shared.

(Read our First-Timer’s Guide to Sante Fe)