CocoaDev

I think that I may have found a bug in NSTreeController or NSOutlineView with bindings but I also might be doing something wrong without knowing it. I made an example project to demonstrate the problem. The example project is at http://www.streambender.com/tmp/NSOutlineViewTest.zip If this is a bug and not a mistake on my part then I’ll report it to Apple. I wanted to solicit opinions here first.

The bug appears to be:

if you add a Core Data NSManagedObject to a managed object context, when there’s an NSOutlineView/NSTreeController observing a bound ‘contentArray’, when the key is programmatically generated by KVC-compliant methods, then some invocation of willChangeValueForKey: for that key will crash down the road. At some non-deterministic time.

if you use an NSTableView/NSArrayController in the same setup as above, no problem.

The sample project is a Core Data project that demonstrates using bindings with an NSTreeController and an NSOutlineView to display data from a Core Data managed context that results from a Spotlight query. I bind contentArray in both controllers to a “relevantItems” key in my app controller. I made my app controller KVC-compliant for “relevantItems” with these methods:

• (NSArray *) relevantItems { // I couldn’t find a safe way to call [genresArrayController selection]. It // crashes the app when there isn’t a selection yet. This method doesn’t. NSArray *aSelections = [genresArrayController selectedObjects]; if([aSelections count] != 1) return nil; NSManagedObject *selectedLocation = [aSelections objectAtIndex:0]; if(!selectedLocation) return nil;

  NSMutableSet *items = [selectedLocation mutableSetValueForKey:@”items”];

  // I can return this as is and bind it to contentSet in a controller or I can do // this next step to turn it into an array and bind it to contentArray. I totally // don’t understand the difference or the pros or cons of either binding. if(!items) return nil; NSArray *aItems = [items allObjects];

  return aItems; }

• (void) setRelevantItems { return; }

I take the results of the NSMetadataQuery and create new Core Data managed objects. I also use bindings in this example project to bind the exact same data through an NSArrayController to an NSTableView that is sitting next to the NSOutlineView. If you un-bind the NSTreeController then everything works fine. In the sample project I have commented out the actual Spotlight query because it isn’t necessary to demonstrate the bug.

When I start the Spotlight query, I use an NSTimer to set up periodic updates:

    queryTimer = [NSTimer scheduledTimerWithTimeInterval:1 target:self
	selector:@selector(updateSearchResults:) userInfo:metadataQuery
	repeats:YES];

[[NSRunLoop currentRunLoop] addTimer:queryTimer forMode:NSDefaultRunLoopMode];

In the update handler, I create NSManagedObjects:

NSManagedObject *selectedGenre = [genresArrayController selection];

NSManagedObject *newItem = [NSEntityDescription
	insertNewObjectForEntityForName:@"Item"
	inManagedObjectContext:managedObjectContext];
[newItem setValue:@"inserted by update" forKey:@"name"];
	
NSMutableSet *items = [selectedGenre mutableSetValueForKey:@"items"];
[items addObject:newItem];

[managedObjectContext processPendingChanges];

Then I manually invoke a KVO update pattern in order to trigger the NSTreeController and the NSOutlineView to call a KVC-compliant method in the application controller that supplies the data for the binding that feeds the controllers:

[self willChangeValueForKey:@"relevantItems"];
[self didChangeValueForKey:@"relevantItems"];

The update message handler adds new objects to the managed object context, which changes the ‘relevantItems’ concept, so I call willChangeValueForKey:@”relevantItems” and didChangeValueForKey:@”relevantItems” so that observers know that the binding changed.

The bug is that after an object has been added to the managed object context from the NSTimer message handler, any time you toggle the UI from one genre to another, something blows up and the app crashes. [BOOM] and I’m looking at a stack trace in somebody else’s code. Apparently it happens during that binding refresh when some observer is updating itself. I think it’s the NSTreeController but the problem might be in the NSOutlineView. I’m not even sure how to tell. Un-bind the NSTreeController and it works fine. The NSArrayController that’s bound to relevantItems and the NSTableView displaying the items work just fine. None of those NSManagedObjects are being deleted or anything at any point so all of the data should be valid at all times.

Am I really looking at a bug in NSTreeController? Or maybe NSOutlineView? Or am I doing something that isn’t supposed to work but just happens to not crash NSArrayController? I don’t want to fall back on using an NSTableView if I’m actually doing something wrong and it might crash on a user some day.

-endymion

Okay, I can simplify this one a little. I was very much under the impression that the problem was that the object was being added in the NSTimer message handler. That is not the case. I added an “add object” button, wired it to the same Core Data code, and it still crashes exactly the same way when you create objects that way. I updated the example project. If you press “add object” in the example project and then toggle back and forth between the two genres, you’ll eventually see the crash. Sometimes you have to jiggle back and forth 10 or 20 times but it will crash.

So none of that Spotlight business has anything to do with it. “If you add a Core Data object when there’s an NSOutlineView/NSTreeController setup already observing a programmatically generated key value then some invocation of willChangeValueForKey: for that key will crash down the road” is as narrow as I can get the problem description now.

I’m either doing something wrong in this code:

NSManagedObject *selectedGenre = [genresArrayController selection];

NSManagedObject *newItem = [NSEntityDescription
	insertNewObjectForEntityForName:@"Item"
	inManagedObjectContext:managedObjectContext];
[newItem setValue:@"test object" forKey:@"name"];
	
NSMutableSet *items = [selectedGenre mutableSetValueForKey:@"items"];
[items addObject:newItem];

[managedObjectContext processPendingChanges];

…or I think there might be a problem with either NSTreeController or NSOutlineView.

Is that about what it should look like when I add a Core Data object that I want to show up in a bound NSTreeController/NSOutlineView? Am I breaking something by binding to those KVC methods for ‘relevantItems’ or something?

I do that in the application where I discovered this problem because I have a lot of context-based filtering to deal with before handing data up to the view. Messing with the fetch predicate on the NSTreeController with it set to ‘Automatically prepares content’ wasn’t cutting it because I can’t represent some of my filters in fetch predicate language. They involve complex many-to-many-to-many set math that I need to be able to do in code. I tried using the @distinctUnionOfArrays operator but it just didn’t work that way, I need to be able to say it in code. If I can just provide a set or an array of objects through a KVC method and then bind an outline view to that then I’m very happy. But it doesn’t seem to work. When I had the NSTreeController set to prepare its own content I didn’t see this bug, even with the above method of adding new objects. I just can’t do it that way because I need more control. Switching to binding the controller to my own KVC methods started this crashing problem.

-endymion

I was struggeling for half a year with all the problems and bugs in NSTreeController until I decided to implement my own tree controller for my project. NSTreeController seems to be very sensitive to the timing and ordering of the change notifications so that it often gets a problem when they derive from undo/redo or from timer calls. My bug reports to Apple of the last four months did not lead to an update of NSTreeController. So this really became a serious issue for our project and I had no other choice than to implement my own controller. And I was very suprised to see that it only took two days to completely implement it. Compared to the endless days of trying to work around the bugs in NSTreeController, it is sad that I started it so late in the game. My tree controller is implemented as a classic data source for an outline view or browser. It can have a set or an array of root objects that are fed via bindings, it uses the same childKeyPath/isLeafKeyPath concept as NSTreeController. Like the Apple’s implementation, it builds a parallel tree structure of its own node objects that observe the original ones for the displayed attributes. I also observe the key paths of the active sort descriptos so that I immediately get an autosorting behavior. In addition to NSTreeController, my nodes know their parent nodes and the tree controller can map the original observed objects back to the tree nodes. Using that I am able to implement the features setSelectedObjects: or uncoverObjects: that Apple’s implementation lacks. It also easy to implement filtering with predicates that takes into account that the whole path to a filtered objects must be visible.

Apple’s basic idea of the tree controller is brilliant but its implementation is closed and buggy. So If you heavily rely on the handling of trees, consider to implement your own tree controller or start a community that does so. It is not hard to do and it enables you to implement fancy features on top of it. When my application will be finished and I have some spare time again I will be happy to share my code with you.

Frank

Wow that sucks to hear. Thanks for your reply, Frank. Ironically enough I got myself into this situation because of Apple so enthusiastically encouraging me to use Objective C, Cocoa, Core Data, and Bindings, in that order. This sucks.

I’ll file a bug report since I have a clear example project to demonstrate it, and then I guess I’ll redesign my UI to not use outline views. I’m a recent switcher, new to Apple development, and I’m not at all interested in reimplementing components that Apple was too lazy to fix before they rolled it out on me. That’s why I left Windows development. This corner that I’m in now feels very familiar and it really sucks.

I guess the lesson here is that all of the bindings technology that Apple released on us isn’t really ready for prime time? I don’t seem to be attempting to do anything too wacky. I wish that something that I read had mentioned that before I spent so much time on a design that depends on bindings. Everybody acts like it ‘just works’ once you get over the learning curve. Newbies take note.

-endymion

I’m using NSTreeController extensively in DL2, and although it’s quirky I’m not experiencing the same bugs you are, which is good news, because it means we can get your code working.

I haven’t done experiments on your code, but the first thing I’m suspicious of is the way -relevantItems is calculated. Why? Because it turns out that Cocoa bindings will crash (randomly!) if you change a bound value without calling -willChange…:/-didChange…:.

“Wait,” you say, “I did call those!” Yes, but here’s the rub: if you do something in your logic to trigger bindings into looking at -relevantItems just before you manually call -didChange…:, or if -relevantItems happens to change on its own at some other time, then you’re still gonna crash.

To see if it’s the latter (-relevantObjects is changing when you don’t expect it): instead of calling [selectedLocation mutableSetValueForKey:], try calling just -valueForKey:. This will return an immutable set. Yes, we hope that calling -allItems on your mutable set rendered it immutable, but I’m not quite that trusting. Better to make sure, because if that set is mutating out from under you, and you aren’t informing your UI of it, you are going to crash.

To see if it’s the former (you are triggering a refresh of the UI after changing the -relevantObjects but before calling didChange…:), instead of calling -processPendingChanges yourself, skip that AND skip calling didChange…:/willChange…:. Instead, register to observe: ‘selectedObjects.items’ in your -awakeFromNib or -init method, and then in your observation just call willChange…:/didChange…:”relevantObjects”.

In this way, you’ll post a notification WHENEVER there’s some change in the selection or items, instead of just when you think there’s a change in selection.

The alternative to this would be to just keep an array as an instance variable that’s a copy of the items you last vended to the NSTreeController, and make sure you only recalculate that array when you’re about to post didChange…: Again, this will ensure that -relevantObjects never returns something different without -didChange…: having been called, first.

Actually, I’m not clear on why you didn’t bind your NSTreeController to “selection.items” in NIB?

-Wil Shipley, Delicious Monster

Wow thanks Wil, I just learned a lot from your reply. I’m kind of new to KVO and bindings and I hadn’t even thought to try observing ‘selection.items’ like that. Didn’t realize it would work. I just tried it and hey wow that’s neat, it works. What you’re saying about me calling will/did when I think that there’s a change as opposed to when it’s actually changing, that makes complete sense now. Unfortunately when I observe ‘selection.items’ and do my will/did from there, it still crashes.

Also thanks for the pointer on calling valueForKey instead of mutableSetValueForKey. I was looking for some way to get an immutable set but I couldn’t find one and I didn’t realize that valueForKey would do it with a many-to-one. Tried that in my relevantItems: method and it works. Still crashes though.

I’m about to try your suggestion to maintain a relevantItems array as an instance variable and bind to that instead of creating the array on the fly. Seems like it might work. This is still looking to me like a valid bug report though because it’s still crashing with those changes.

And FYI, I couldn’t bind the NSTreeController to “selection.items” in the NIB because it unfortunately isn’t that simple. I need to filter the items based on the states of some filter UI elements. I’m implementing an iTunes-like data browser UI.

Thanks so much for everybody’s time!

-endymion

Okay update: binding to an array that I maintain didn’t help, same crash still. Instead of calculating the ‘relevantItems’ key on-demand in the KVC methods listed above, I changed it to:

• (NSArray *) relevantItems { return relevantItems; }

• (void) setRelevantItems { return; }

• (void) determineRelevantItems { // I couldn’t find a safe way to call [locationTreeController selection] NSArray *aSelections = [locationTreeController selectedObjects]; if([aSelections count] != 1) return nil; NSManagedObject *selectedLocation = [aSelections objectAtIndex:0]; if(!selectedLocation) return nil;

  NSMutableSet *items = [selectedLocation mutableSetValueForKey:@”items”];

  if(!items) return nil; NSArray *aItems = [items allObjects];

  relevantItems = aItems; }

…with an NSArry *relevantItems instance variable. I added an observer for the genre controller’s ‘selection.items’ and I alter the relevantItems instance variable there, surrounded by will/did:

• (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context { …

  if(object == locationTreeController && [keyPath isEqual:@”selection.items”]) { [self willChangeValueForKey:@”relevantItems”]; [self determineRelevantItems]; [self didChangeValueForKey:@”relevantItems”]; }

Works great! Very educational experience. Unfortunately the thing still crashes in the same exact way.

Is the fundamental problem here that I’m making a change to the relevantItems array that doesn’t happen between willChangeValueForKey: and didChangeValueForKey: ? I’m getting the impression that it has something to do with that? I’m trying to be careful right now to only make data model modifications that are bracketed by those methods but that isn’t working. But that would cause something like this, theoretically?

-endymion

I constructed a very simple CoreData test case for demonstrating the problems NSTreeController has with the ordering of change notifications even when the changes are properly wrapped by willChange/didChange calls:

http://homepage.mac.com/illenberger/.Public/TreeControllerTest.zip

In this example all change notifications are automatically generated by the NSManagedObject default implementation.

Frank

Thanks for posting that again Frank, I didn’t manage to spot your example in my own research. Your example project is much simpler and better documented than mine, I’m humbled. A few times that I ran your example I ended up seeing the same sort of [NSCFArray objectAtIndex:]: index (2) beyond bounds (2) exception that I see with my example. Other times it breaks in other ways. It’s interesting that your example doesn’t even add or delete objects to the managed object context at all. It just adjusts relationships.

Oh well, I was really hoping that I was doing something wrong and that this would become a learning experience for me. I have learned a LOT from your input and Wil’s, but I guess I really am looking at a bug in NSTreeController. The squeaky wheel gets the grease, so I have filed my own bug report to Apple on this, ticket number 4476512.

Now I’d like to try to find a workaround for this that will allow people to use NSTreeController with bindings, without each individual developer being forced to implement their own NSTreeController. I have spent enough of my time re-implementing broken and missing things in Microsoft’s MFC that I’m not about to start going down that road fixing Cocoa before I use it.

My example project demonstrates a method-generated contentSet binding, and Franks’ demonstrates binding NSTreeContoller’s contentObject to an instance variable. Both avenues fail. It looks to me like NSTreeController simply will not work with bindings unless you use ‘Automatically prepares content’. You simply can’t provide content to an NSTreeController through bindings if the content will change at any point. Is that the consensus?

The only way that I have so far seen NSTreeController behave properly when used with bindings is when it prepares its own content. I can’t use that in my particular project because I can’t express the content set as a fetch predicate. I can add a fetch predicate string to NSTreeController in the NIB but I can’t use that route because deriving my particular content set is just too hard for a fetch predicate to handle. The @distinctUnionOfArrays operator is either broken or it isn’t documented well enough for me to use it. I also see other reports of NSTreeController exploding unexpectedly when you programmatically change the fetch predicate, so I’m almost afraid to even bother trying to change the predicate at runtime.

So is there some way for me to hack my own custom methods into a fetch predicate? Does the invocation of a fetch predicate by the controller potentially lead to the invocation of some kind of KVC methods in one of my objects that I could override or provide in order to do the set math item filtering that I require? Like can I add a custom “isRelevant” operator or something? A method on an object? “Item.isRelevant == true” on a custom NSManagedObject subclass for my “Item” entity perhaps? I don’t quite understand the fetch mechanism enough to understand whether that would be a Bad Thing from a performance standpoint? Perhaps I could pre-generate a relevantItems array and then check quickly in an isRelevant: method, but is that just a bad idea in the first place? That smells like it’s becoming an O(n^2) operation because of an O(n) set inclusion check within an O(n) isRelevant: check from the Core Data fetch that NSTreeContoller would run?

Thanks again for your help, Frank and Wil. I’m learning a lot from all of this regardless of whether it’s my code or Apple’s that’s malfunctioning.

-endymion

Wil, an update in regard to your suggestion to look into avenues that make use of simply binding the NSTreeController to ‘selection.items’ from the other controller. I just tried that with the contentSet binding and before I could even get into the problem of filtering the items, it crashes. Same deal as before. The act of adding an object to the managed object context is what triggers it. So no O(n) solution to the item filtering issue will help because NSTreeController apparently just can’t function from any of those “contentXXXXXX” bindings. They just don’t seem to work, none of them. It will work fine with ‘Automatically prepares content’, but not with content supplied by a binding.

Oh well, good suggestion anyway. Thanks!

-endymion

Workaround attempt update: I just tried explicitly providing content to the NSTreeController by using the controller object’s setContent: method. I calculate my (immutable) relevantItems array with this method:

• (NSArray *) relevantItems { // I couldn’t find a safe way to call [locationTreeController selection] NSArray *aSelections = [locationTreeController selectedObjects]; if([aSelections count] != 1) return nil; NSManagedObject *selectedLocation = [aSelections objectAtIndex:0]; if(!selectedLocation) return nil;

  NSSet *items = [selectedLocation valueForKey:@”items”];

  if(!items) return nil; NSArray *aItems = [items allObjects];

  return aItems; }

After I add a new Item object to the managed object context, I hand one of these arrays to the tree controller with its setContent: method as so:

[itemsTreeController setContent:[self relevantItems]];

No dice though, same exact crash. After adding an object to the managed object context, the app will always crash down the road as you toggle between selections in the locations controller. It behaves the same as described above, where it doesn’t necessarily crash immediately after you change the model data but it will eventually crash down the road at some point. Always, every time.

So I guess the problem isn’t specific to bindings, or perhaps there are multiple bugs in NSTreeController.

-endymion

The above workaround attempt with NSTreeController:setContent: does work if I do this first:

[itemsTreeController setContent:nil];
[itemsTreeController setContent:[self relevantItems]];

Presumably what I’m doing here is preventing NSTreeController from ever having a chance to fumble the ball by putting blinders on it when I change data in the model. This sucks immensely for me because my objects are added by a Spotlight query, so on every update my user’s current selection in itemsTreeController will vanish as I set its content to nil. Very stinky, but this workaround does work.

-endymion

Okay, here’s a final workaround for this problem that actually does work. I’m posting this for anybody who runs into problems with this before Apple fixes NSTreeController.

In the class that’s handling all of this logic (MyDocument in an XCode “Core Data Document-Based Application with Spotlight Importer”) I have added a relevantItems instance variable that I manage on my own:

NSMutableArray *relevantItems;

I feed that NSMutableArray to my NSTreeController using NSTreeController:setContent, which works. When my Spotlight update method adds a new file, I add the file to the managed object context and then immediately and explicitly I add the new object to that mutable array:

NSManagedObject *newItem = [NSEntityDescription
	insertNewObjectForEntityForName:@"Item"
	inManagedObjectContext:managedObjectContext];

// Add the child to the relevantItems array that I have to maintain
// becausee of a bug in NSTreeController that makes it crash if you
// change data from under it.
[relevantItems addObject:newItem];

That mechanism allows me to add new objects to my data in a way that preserves the user’s selection in the outline view.

Then any time locationTreeController’s selection changes, which changes the concept of which items are relevant, I do this:

• (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context { …

  if(object == locationTreeController && [keyPath isEqual:@”selection.items”]) { // Put blinders on NSTreeController so it won’t crash when // I change its content. [itemsTreeController setContent:nil];

    // Ditch the old content array and get a new empty one.
    [relevantItems release];  relevantItems = [[NSMutableArray alloc] init];
  
    // Do the crazy set math that determines relevant items.
    [relevantItems addObjectsFromArray:[self relevantItems]];
  
    // Hand the new array of relevant items to the controller.
    [itemsTreeController setContent:relevantItems];   }
  

This mechanism works great as a workaround for the crash until there’s a fix.

-endymion

Bah. I spoke too soon. If you have an NSMutableArray and you use NSTreeController:setContent to give the array to the controller, and then later you add a new object to the array with NSMutableArray:addObject, the object will not appear in the controller. I briefly misled myself into thinking that the above workaround was doing the trick but it was actually releasing and re-creating the entire array every time I added an object because I was observing “selection.items” as opposed to just “selection”. So any added object triggered the purge/recreation of the relevantItems array. Pffffttthhhhhppppfpfpfffff…

This all sucks so much that I’m going to stop using NSTreeController until Apple fixes it. Which means no outline view in my application whether my design wants one or not.

I figured that after days and days of investigation I would at least be able to post a workaround for other people to use but nope. Sorry…

-endymion

Maybe there is one last straw… This is a hack I used before I threw NSTreeController out of my application. It is bad, uses posing for private classes. But it helped me around some of the exceptions.

@interface _NSArrayControllerTreeNode : NSObject { id _observedObject; NSArrayController *_controller; NSTreeController *_treeController; NSMutableArray *_subNodes; _NSArrayControllerTreeNode *_parentNode; int _NSArrayControllerTreeNodeFlags; }

• (void)updateSubnodes; @end

@interface MEArrayControllerTreeNode : _NSArrayControllerTreeNode { } @end

@implementation MEArrayControllerTreeNode

• (void)updateSubnodes { if([self count]) [self subnodeAtIndex:0]; [super updateSubnodes]; }

@end

// Put the posing initialization somewhere early [MEArrayControllerTreeNode poseAsClass:[_NSArrayControllerTreeNode class]];

Frank

Thanks for digging that up for me, Frank. I’m pretty sour on using NSTreeController now but wow that’s pretty neat. I had actually never even seen poseAsClass: used for anything before. Very educational. I have learned a lot from this whole thing. I’m going to spend the rest of the afternoon reading about that, now that I’ve ripped the NSOutlineView out of my app entirely.

-endymion

I looked into this issue as well a bit. It appears to be a defect within the caching of NSTreeController. It also appears (from the topic NSTreeControllerBug) that this has gotten worse with 10.4.3. In that topic, they solve the problem by wrapping setContent to break all caching, the same way endymion did above. However, Frank’s example still breaks with their workaround, as the updateManager rolls changes in directly. It appears that there is no support for the structure of a tree to change via object updates - it seems that all structural changes other than deleting/inserting objects will break the controller’s cache. It appears Frank actually tried to make more invasive cuts into the implementation to solve KVO/NSUndoManager style-changes in addition to the MOC notification-based changes which directly hit setContent.

-DW

Wil, I can reproduce Frank’s problem here. Do you have a workaround?

Frank, How did you implement your own tree controller? Is it a subclass of NSController or NSObjectController? How did you handle the selection proxy?

I’m seeing random crashes with NSTreeController and undo/redo – I am about to bail on it as well. This is Not Ready For Prime Time, as far as I can tell.

Using Frank’s hack above, I was able to get a stable NSTreeController setup. See the following for more info: http://www.cocoabuilder.com/archive/message/cocoa/2006/7/6/166974 or http://lists.apple.com/archives/cocoa-dev/2006/Jul/msg00241.html

-George

Why do you send the messages willChangeValueForKey and didChangeValueForKey right after each other? To me, someone new on KVO, it seems logical to send the first message, then do the change, and then send the second message…

Hendrik

Under Leopard, the problem seems to be resolved by activating the garbage collector.

Francis